Add SAS API interface #729

bolyu · 2024-02-02T19:40:05Z

Add API interface of SAS solvers.

CLAassistant · 2024-02-02T19:40:12Z

All committers have signed the CLA.

pchtsp

Thanks for the contribution. Please find some comments. I haven't checked everything but this should already be a good start.

I've also seen that the two python packages are available through pypi.

Can you add the solver to the github CI? example:

pulp/.github/workflows/pythonpackage.yml

Line 32 in 6e18ecb

- name: Install gurobipy

?
Can you add the solver to the testing suite? example:

pulp/pulp/tests/test_pulp.py

Line 1576 in 6e18ecb

class MOSEKTest(BaseSolverTest.PuLPTest):

?

pchtsp · 2024-02-27T10:32:47Z

pulp/apis/sas_api.py

+                msg=msg,
+            )
+            self._sas = saspy.SASsession()
+            self.warmStart=warmStart


warmstart can be passed to LpSolver and then becomes available inside self.optionsDict. See how it's used in other solvers. No need to assign it to the object.

pchtsp · 2024-02-27T10:33:34Z

pulp/apis/sas_api.py

+            self.warmStart=warmStart
+            # Only named options are allowed in SAS solvers.
+            self.solverOptions = {key:value for key, value in solverParams.items()}
+        def available(self):


I believe you need to run the black linter on the code.
https://coin-or.github.io/pulp/develop/contribute.html#applying-the-black-linter-formatter

pchtsp · 2024-02-27T10:34:50Z

pulp/apis/sas_api.py

+
+            return (f"{name}-pulp.{n}" for n in args)
+
+        def _delete_tmp_files(self, *args):


there already exists a delete_tmp_files method that does almost exactly this. No need to create a new one. The same goes for _create_tmp_files

pchtsp · 2024-02-27T10:37:13Z

pulp/apis/sas_api.py

+            decompsubprob_str = self._create_statement_str("decompsubprob")
+            rootnode_str = self._create_statement_str("rootnode")
+
+            if lp.isMIP():


simpler:

if lp.isMIP() and not self.mip: warnings.warn("SAS94 will solve the relaxed problem of the MILP instance.")

pchtsp · 2024-02-27T10:38:26Z

pulp/apis/sas_api.py

+                        primalin=tmpMst,
+                        postfix=postfix,
+                    )
+                    self.solverOptions["primalin"] = f"primalin{postfix}"


it's not a good practice to edit the solverOptions attribute. You should avoid editing during solve. It should be stateless.

pchtsp · 2024-02-27T10:41:07Z

pulp/apis/sas_api.py

+            self._delete_tmp_files(tmpMps, tmpMst)
+
+            # Store log and ODS output
+            self._log = res["LOG"]


I do not see this property being defined anywhere: _log. Why do we need to store it inside self?

pchtsp · 2024-02-27T10:41:32Z

pulp/apis/sas_api.py

+            self._log = res["LOG"]
+            if self.msg:
+                print(self._log)
+            self._macro = dict(


same goes for _macro. is it needed after the solving? why?

_macro is used to store some information returned by SAS solvers. It has information like primal and dual infeasibility, and SAS solution status. It might be interesting to SAS users.

pchtsp · 2024-02-27T10:42:15Z

pulp/apis/sas_api.py

+                raise PulpSolverError("PuLP: Error ({err_name}) \
+                        while trying to solve the instance: {name}".format(
+                        err_name=self._macro.get("STATUS", "ERROR"), name=lp.name))
+            status = self._read_solution(lp, primal_out, dual_out)


what if there are no values available? should we read the soluton still?

If no values available, the solution will be NaN.

pchtsp · 2024-02-27T10:43:56Z

pulp/apis/sas_api.py

+                except FileNotFoundError:
+                    pass
+
+        def _write_sol(self, filename, vs):


there are many methods that are common between the two solvers. Could we just inherit one from the other and get all those methods?

pchtsp · 2024-02-27T10:46:38Z

pulp/apis/sas_api.py

+                raise PulpSolverError("SASCAS : Objective sense should be min or max.")
+
+            status = None
+            with redirect_stdout(SASLogWriter(self.msg)) as self._log_writer:


this part has many indentations and ifs. Can it be refactored so that it's easier to read? for example breaking it into other methods. re-using the code with different arguments, etc.

pchtsp · 2024-03-21T16:36:12Z

Hello, I solved the conflicts with master. Now the tests are failing because you need to run the black linter as I previously said:

https://coin-or.github.io/pulp/develop/contribute.html#applying-the-black-linter-formatter

pchtsp

Thanks.

I've left more comments again. You need to get your code to run in the pulp CI, so fix the black linter first.

If someone else from SAS can help you reviewing the python code before asking for a review here, that would be awesome. I do not have too much time to give the amount of feedback/ improvements that is needed for this PR.

pchtsp · 2024-03-21T16:39:15Z

pulp/apis/sas_api.py

+
+    def _create_statement_str(self, statement):
+        """Helper function to create the strings for the statements of the proc OPTLP/OPTMILP code."""
+        stmt = self.solverOptions.pop(statement, None)


please do not edit this property, it should be static. There are many places where it's edited in the code.

pchtsp · 2024-03-21T16:45:25Z

pulp/apis/sas_api.py

+            except:
+                return False
+
+        def toDict(self):


(1) you have this function duplicated.
(2) you should not just copy the inherit method, you should call the LpSolver_CMD.toDict method and then add the functionality that you need.
(3) What is cas object and how do you expect to serialize it?

pchtsp · 2024-03-21T16:46:19Z

pulp/apis/sas_api.py

+            return True
+
+        def sasAvailable(self):
+            if self.cas == None:


self.cas is None instead

phchri · 2024-09-16T10:02:38Z

I improved the PuLP interface with lessons learned from other interfaces, added the "with" options that other interfaces support (and a test for it) and improved performance of reading the output. @bolyu: Please review and test the changes.

bolyu · 2024-09-16T16:33:11Z

I have reviewed and tested the changes. The updates look good to me.

pchtsp · 2024-09-19T15:21:29Z

thanks to both. The CI tests are still failing. see below. Are you sure the code works when there's no SAS solver around?
If no SAS solver is available, the solver should return available()=False, which doesn't seem to be the case because the libraries are installed.

 SAS process has terminated unexpectedly. Pid State= (2219, 64000)
Traceback (most recent call last):

  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pulp/apis/sas_api.py", line 233, in __init__
    self.sas = saspy.SASsession(**self._saspy_options)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/saspy/sasbase.py", line 590, in __init__
    self._io = SASsessionSTDIO(sascfgname=self.sascfg.name, sb=self, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/saspy/sasiostdio.py", line 291, in __init__
    self._startsas()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/saspy/sasiostdio.py", line 512, in _startsas
    ll = self.submit("options svgtitle='svgtitle'; options validvarname=any validmemname=extend; ods graphics on;", "text")
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/saspy/sasiostdio.py", line 985, in submit
    raise SASIOConnectionTerminated(Exception)
saspy.sasexceptions.SASIOConnectionTerminated: No SAS process attached. SAS process has terminated unexpectedly.
<class 'Exception'>

phchri · 2024-09-19T19:00:19Z

My bad, I assumed your machines would not have the SAS packages installed. I'll come up with another way of doing this next week.

phchri · 2024-09-23T12:42:29Z

@bolyu I updated the interface classes, please review and verify the code on your side.

pulp/apis/sas_api.py

phchri · 2024-09-24T08:40:22Z

@bolyu I adjust the code as you suggested.

bolyu · 2024-09-24T14:06:09Z

@bolyu I adjust the code as you suggested.

The current version looks good.

@pchtsp Please rerun the CI tests. Thank you so much!

bolyu added 4 commits January 19, 2024 12:17

Add SAS API interface

d0b6dbd

Update the version

6f06de5

Remove unused comments

a164338

Add an empty line to the end of the file

ef96a52

pchtsp requested changes Feb 27, 2024

View reviewed changes

bolyu and others added 3 commits March 5, 2024 08:53

Update SAS API interface

f22dfe4

Update documentation

83218cc

Merge branch 'master' into master

33dceba

pchtsp requested changes Mar 21, 2024

View reviewed changes

bolyu and others added 5 commits April 10, 2024 12:44

Update SAS API

c597f1d

Merge branch 'coin-or:master' into master

c84e13d

Update SAS API interface (need more changes)

a028743

Various improvements to the SAS interfaces

087b8ef

Merge branch 'master' into master

d4df18a

bolyu requested a review from pchtsp September 18, 2024 16:18

allow SAS solver classes without valid connection

ade07e5

bolyu commented Sep 23, 2024

View reviewed changes

pulp/apis/sas_api.py Show resolved Hide resolved

pulp/apis/sas_api.py Outdated Show resolved Hide resolved

disable printing and fix comment

86e712c

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add SAS API interface #729

Add SAS API interface #729

bolyu commented Feb 2, 2024

CLAassistant commented Feb 2, 2024 •

edited

Loading

pchtsp left a comment

pchtsp Feb 27, 2024

pchtsp Feb 27, 2024

pchtsp Feb 27, 2024

pchtsp Feb 27, 2024

pchtsp Feb 27, 2024

pchtsp Feb 27, 2024

pchtsp Feb 27, 2024

bolyu Mar 5, 2024

pchtsp Feb 27, 2024

bolyu Mar 5, 2024

pchtsp Feb 27, 2024

pchtsp Feb 27, 2024

pchtsp commented Mar 21, 2024

pchtsp left a comment

pchtsp Mar 21, 2024

pchtsp Mar 21, 2024

pchtsp Mar 21, 2024

phchri commented Sep 16, 2024

bolyu commented Sep 16, 2024

pchtsp commented Sep 19, 2024

phchri commented Sep 19, 2024

phchri commented Sep 23, 2024

phchri commented Sep 24, 2024

bolyu commented Sep 24, 2024


		return (f"{name}-pulp.{n}" for n in args)

		def _delete_tmp_files(self, *args):

Add SAS API interface #729

Are you sure you want to change the base?

Add SAS API interface #729

Conversation

bolyu commented Feb 2, 2024

CLAassistant commented Feb 2, 2024 • edited Loading

pchtsp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pchtsp commented Mar 21, 2024

pchtsp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

phchri commented Sep 16, 2024

bolyu commented Sep 16, 2024

pchtsp commented Sep 19, 2024

phchri commented Sep 19, 2024

phchri commented Sep 23, 2024

phchri commented Sep 24, 2024

bolyu commented Sep 24, 2024

CLAassistant commented Feb 2, 2024 •

edited

Loading