NDOSolver / FiOracle
Interfaces and Solvers for NonDifferentiable Optimization
|
Public Member Functions | |
LukFiOrcl (std::istream *iStrm=nullptr) | |
![]() | |
virtual NDOSolver * | GetNDOSolver (void) |
FiOracle (void) | |
virtual void | SetNDOSolver (NDOSolver *NwSlvr=0) |
virtual void | SetFiLog (ostream *outs=0, const char lvl=0) |
virtual void | SetFiTime (const bool TimeIt=true) |
virtual void | SetMaxName (cIndex MxNme=0) |
virtual Index | GetNumVar (void) const |
virtual Index | GetMaxNumVar (void) const |
virtual Index | GetNrFi (void) const |
virtual Index | MaxNConst (void) |
virtual bool | IsFiContinuous (cIndex NrFi=Inf< Index >()) |
virtual bool | IsFiConvex (cIndex NrFi=Inf< Index >()) |
virtual bool | HasGi (cIndex NrFi=Inf< Index >()) |
virtual bool | IsGiContinuous (cIndex NrFi) |
virtual bool | HasH (cIndex NrFi) |
virtual bool | IsHContinuous (cIndex NrFi) |
virtual Index | GetMaxName (void) const |
virtual HpNum | GetMinusInfinity (void) |
virtual Index | GetMaxNZ (cIndex wFi=Inf< Index >()) const |
virtual Index | GetMaxCNZ (cIndex wFi=Inf< Index >()) const |
virtual bool | GetUC (cIndex i) |
virtual LMNum | GetUB (cIndex i) |
virtual LMNum | GetBndEps (void) |
virtual HpNum | GetGlobalLipschitz (cIndex wFi=Inf< Index >()) |
virtual Index | GetBNC (cIndex wFi) |
virtual Index | GetBNR (cIndex wFi) |
virtual Index | GetBNZ (cIndex wFi) |
virtual void | GetBDesc (cIndex wFi, int *Bbeg, int *Bind, double *Bval, double *lhs, double *rhs, double *cst, double *lbd, double *ubd) |
virtual Index | GetANZ (cIndex wFi, cIndex strt=0, Index stp=Inf< Index >()) |
virtual void | GetADesc (cIndex wFi, int *Abeg, int *Aind, double *Aval, cIndex strt=0, Index stp=Inf< Index >()) |
virtual void | SetLambda (cLMRow Lmbd=0) |
virtual void | SetLamBase (cIndex_Set LmbdB=0, cIndex LmbdBD=0) |
virtual bool | SetPrecision (HpNum Eps) |
virtual bool | NewGi (cIndex wFi=Inf< Index >()) |
virtual void | SetGiName (cIndex Name) |
virtual FiStatus | GetFiStatus (Index wFi=Inf< Index >()) |
void | FiTime (double &t_us, double &t_ss) |
double | FiTime (void) |
virtual void | Deleted (cIndex i=Inf< Index >()) |
virtual void | Aggregate (cHpRow Mlt, cIndex_Set NmSt, cIndex Dm, cIndex NwNm) |
virtual | ~FiOracle () |
Reading the data of the problem | |
Index | NameF |
name (number) of the function at point Lambda | |
Index | NrCmp |
number of component functions | |
Index | seed |
seed for random number generation | |
Row | bQR |
Row | aQR |
LMMat | cQR |
void | GetInitialPoint (LMRow lmb) |
Index | GetName (void) |
virtual HpNum | Fi (cIndex wFi=Inf< Index >()) |
virtual Index | GetGi (SgRow SubG, cIndex_Set &SGBse, cIndex Name=Inf< Index >(), cIndex strt=0, Index stp=Inf< Index >()) |
virtual HpNum | GetVal (cIndex Name=Inf< Index >()) |
virtual HpNum | GetLowerBound (cIndex wFi=Inf< Index >()) |
void | SetDimension (void) |
Additional Inherited Members | |
![]() | |
enum | FiStatus { kFiNorm = 0 , kFiError , kFiChgd , kFiStop , kFiCont } |
![]() | |
NDOSolver * | Slvr |
Index | NumVar |
(current) number of variables if Fi() | |
Index | MaxName |
maximum name to be used in SetGiName() | |
cLMRow | Lambda |
cIndex_Set | LamBase |
Index | LamBDim |
length of LamBase[] | |
bool | LHasChgd |
ostream * | FiLog |
the output stream object for log purposes | |
char | FiLLvl |
the "level of verbosity" of the log | |
OPTtimers * | Fit |
OPTtimer for timing purposes. | |
LukFiOrcl | ( | std::istream * | iStrm = nullptr | ) |
Constructor of the class. The parameter ‘iStrm’, if provided, is taken as a pointer to a istream from which the algorithmic parameters for the subgradient algorithm are sequentially read in the following order. Each parameter must be placed at the beginning of a separate line, max 255 characters long, with all the rest of the line up to the first newline character (apart from a separating whitespace) being available for comments. Any line whose first character is '#' and any blank line is ignored. If 0 is passed, the file ends before reaching a given parameter, or some parameter is in the wrong format, each non-specified parameter is given a default value, shown in [] below.
NameF [1]: the function that the FiOracle returns, chosen in the list of available 25 ones below
NrVr [2]: a few functions of the list, namely 26, 27 and 28, have a parametric number of variable, that is fixed to NrVr
NrCmp [2]: function 28 (MaxQR) has a parametric number of components, i.e., it is the sum of NrCmp functions
seed [1]: function 28 (MaxQR) is randomly generated, with this seed
Here comes the list of functions:
1 Rosenbrock. [ NonConvex ]. Optimal point \(\lambda^* = (1, 1)\). Optimal value \(f^* = 0\).
\[f(\lambda) = 100(\lambda_2-\lambda_1^2)^2+(1-\lambda_1)^2 \]
See "Nonsmooth Optimization", M. Makela and P. Neittaanmaki, (1992)
2 Crescent. [ NonConvex ]. Optimal point \(\lambda^* = (0, 0)\). Optimal value \(f^* = 0\).
\[f(\lambda) = \max\left\{ \lambda_1^2+(\lambda_2-1)^2 + \lambda_2 - 1\, ,\,-\lambda_1^2-(\lambda_2-1)^2 + \lambda_2 + 1 \right\} \]
See "Nonsmooth Optimization", M. Makela and P. Neittaanmaki, (1992)
3 CB2 (Charalambous/Bandler). [ Convex ]. Optimal point \(\lambda^* = (1.139286, 0.899365)\). Optimal value \(f^* = 1.9522245\).
\[f(\lambda) = \max\left\{\lambda_1^2+\lambda_2^4\,,\, (2-\lambda_1)^2 + (2-\lambda_2)^2\,,\,2e^{-\lambda_1+\lambda_2} \right\} \]
See "Nonsmooth Optimization", M. Makela and P. Neittaanmaki, (1992)
4 CB3 (Charalambous/Bandler). [ Convex ]. Optimal point \(\lambda^* = (1, 1)\). Optimal value \(f^* = 2\).
\[f(\lambda) = \max\left\{\lambda_1^4+\lambda_2^2\,,\, (2-\lambda_1)^2 + (2-\lambda_2)^2\,,\, 2e^{-\lambda_1+\lambda_2} \right\} \]
See "Nonsmooth Optimization", M. Makela and P. Neittaanmaki, (1992)
5 DEM (Demyanov/Malozemov). [ Convex ]. Optimal point \(\lambda^* = (0,-3)\). Optimal value \(f^* = -3\)
\[f(\lambda) = \max\left\{5\lambda_1+\lambda_2\,,\, -5\lambda_1+ \lambda_2\,,\,\lambda_1^2+\lambda_2^2+4\lambda_2 \right\} \]
See "Nonsmooth Optimization", M. Makela and P. Neittaanmaki, (1992)
6 QL. [ Convex ]. Optimal point \(\lambda^* = (1.2, 2.4)\). Optimal value \(f^* = 7.2\).
\[f(\lambda) = \max\left\{\lambda_1^2+\lambda_2^2\,,\, \lambda_1^2+ \lambda_2^2+ 10(-4\lambda_1-\lambda_2+4)\,,\,\lambda_1^2+\lambda_2^2 + 10(-\lambda_1-2\lambda_2+6) \right\} \]
See "Nonsmooth Optimization", M. Makela and P. Neittaanmaki, (1992)
7 LQ. [ Convex ]. Optimal point \(\lambda^* = (\frac{1}{\sqrt 2}, \frac{1}{\sqrt 2})\). Optimal value \(f^* = -\sqrt 2\).
\[f(\lambda) = \max\left\{-\lambda_1-\lambda_2\,,\, -\lambda_1- \lambda_2+(\lambda_1^2+\lambda_2^2-1)\right\} \]
See "Nonsmooth Optimization", M. Makela and P. Neittaanmaki, (1992)
8 Mifflin1. [ Convex ]. Optimal point \(\lambda^* = (1, 0)\). Optimal value \(f^* = -1\).
\[f(\lambda) = -\lambda_1+20\max\left\{\lambda_1^2+\lambda_2^2-1 \,, \, 0\right\} \]
See "Nonsmooth Optimization", M. Makela and P. Neittaanmaki, (1992)
9 Mifflin2. [ NonConvex ]. Optimal point \(\lambda^* = (1, 0)\). Optimal value \(f^* = -1\).
\[f(\lambda) = -\lambda_1+2(\lambda_1^2+\lambda_2^2-1) + 1.75\left|\lambda_1^2+\lambda_2^2-1\right| \]
See "Nonsmooth Optimization", M. Makela and P. Neittaanmaki, (1992)
10 Wolfe. [ NonConvex ]. Optimal value \(f^* = -8\)
\[ f(\lambda) = \left \{ \begin{array}{ll} 5\sqrt{9\lambda_1^2+16\lambda_2^2} & \lambda_1\geq|\lambda_2| \\ 9\lambda_1+16|\lambda_2|& 0<\lambda_1<|\lambda_2|\\ 9\lambda_1+16|\lambda_2|-\lambda_1^9&\lambda_1\leq 0 \end{array} \right. \]
See "Test problems for nonsmooth unconstrained optimization and linearity constrained optimization". Technical Report 798, Institute of Computer Science, Academy of Science of the Czech Republic, (2000)
11 Rosen. [ Convex ]. Optimal point \(\lambda^* = (0, 1, 2, -1)\). Optimal value \(f^* = -44\)
\[f(\lambda) = \max\left\{ f_1(\lambda) \,,\, f_1(\lambda)+10f_2(\lambda)\,,\, f_1(\lambda)+10f_3(\lambda)\,,\, f_1(\lambda)+10f_4(\lambda)\right\}, \]
where
\[\left | \begin{array}{ll} f_1(\lambda)=& \lambda_1^2+\lambda_2^2+2\lambda_3^2+\lambda_4^2 -5\lambda_1-5\lambda_2-21\lambda_3+7\lambda_4 \\ f_2(\lambda)=& \lambda_1^2+\lambda_2^2+\lambda_3^2+\lambda_4^2 +\lambda_1-\lambda_2+\lambda_3-\lambda_4-8\\ f_3(\lambda)=& \lambda_1^2+2\lambda_2^2+\lambda_3^2+2\lambda_4^2 -\lambda_1-\lambda_4-10\\ f_4(\lambda)=& \lambda_1^2+\lambda_2^2+\lambda_3^2 +2\lambda_1-\lambda_2-\lambda_4-5 \end{array} \right. \]
See "Nonsmooth Optimization", M. Makela and P. Neittaanmaki, (1992)
12 Shor. [ Convex ]. Optimal point \(\lambda^* = (1.12434, 0.97945, 1.47770, 0.92023, 1.12429)\). Optimal value \(f^* = 22.60016\).
\[f(\lambda) = \max_{1\leq i \leq 10} \left\{ b_i \sum_{j=1}^5 ( \lambda_{j} - a_{ij} )^2 \right\} \]
See "Methods of Descent for Nondifferentiable Optimization", K. Kiwiel, Lectures Notes in Mathematics, A. Dold and B. Eckmann Eds. Springer-Verlag, (1985)
13 Maxquad (by Lemarechal). [ Convex ]. Optimal point \(\lambda^* = (-0.1263, -0.0346, -0.0067, -0.2668, 0.0673, 0.2786, 0.0744, 0.1387, 0.0839, 0.0385)\). Optimal value \(f^* = -0.8414084\).
\[f(\lambda) = \max_{1\leq k \leq 5} \left( \lambda^TA_k \lambda - b_k ^T\lambda \right) \]
See "Nonsmooth Optimization", Proceedings of a IISA Workshop, C. Lemarechal and R. Mifflin, Eds. Vol. 3 (1977)
14 Maxq. [ Convex ]. Optimal point \(\lambda^*= 0\). Optimal value f* = 0.
\[f(\lambda) = \max_{1\leq i \leq 20} \lambda_i^2 \]
See "Nonsmooth Optimization", M. Makela and P. Neittaanmaki, (1992)
15 Maxl. [ Convex ]. Optimal point \(\lambda^*= 0\). Optimal value \(f^* = 0\).
\[f(\lambda) = \max_{1\leq i \leq 20} |\lambda_i| \]
See "Nonsmooth Optimization", M. Makela and P. Neittaanmaki, (1992)
16 TR48 (by Goffin). [ Convex ]. Optimal point \(\lambda^*\) = (144, 257, 0, 483, 89, -165, -72, -252, -88, -178, 311, 126, 7, -135, 158, 209, 101, -92, 229, 80, 95, 71, -244, 102, -12, 132, 337, 61, 104, 41, 261, 118, 99, -246, 156, -270, 330, -130, 952, -62, 161, 484, 122, 474, 1086, 861, -170, 206). Optimal value \(f^* = -638565\).
\[f(\lambda) = \sum_{j=1}^{48} d_j\max_{1\leq i\leq 48}(\lambda_i-a_{ij}) - \sum_{i=1}^{48} s_i\lambda_i \]
See "Nonsmooth Optimization", Proceedings of a IISA Workshop, C. Lemarechal and R. Mifflin, Eds. Vol. 3 (1977)
17 Colville1. [ NonConvex ]. Optimal value \(f^* = -32.348679\).
18 HS78. [ NonConvex ]. Optimal value \(f^* = -2.9197004\).
19 El-Attar. [ NonConvex ]. Optimal value \(f^* = -0.5598131\).
See "An algorithm for l1-norm minimization with application to nonlinear
l1-approximation", R.A. El-Attar, M. Vidyasagar and S.R.K. Dutta, Siam Journal Numeircal Analysis, 16(1), pp. 70-86, (1979)
20 Gill. [ NonConvex ]. Optimal value \(f^* = 9.7857721\).
21 Steiner2. Optimal value \(f^* = 16.703838\).
22 Goffin. [ Convex ]. Optimal value \(f^* = 0\).
See "Nonsmooth Optimization", M. Makela and P. Neittaanmaki, (1992)
23 MXHILB. [ NonConvex ]. Optimal value \(f^* = 0\).
24 L1HILB. [ NonConvex ]. Optimal value \(f^* = 0\).
25 Shell Dual. [ NonConvex ]. Optimal value \(f^* = 32.348679\).
26 smooth
27 AbsVal
28 MaxQR
29 Lewis
@ --------------------------------------------------------------------—
Implements FiOracle.
|
virtual |
GetGi() [and GetVal(), see below] can be used to query information about the items. The method allows to access both the newly produced item corresponding to the last call to NewGi() [see above], if it has not been "named" yet [see SetGiName() below], and all "named" items recorded in the FiOracle memory, if any [see SetMaxName() above]. This is specified by the value of Name. If Name < n, where n is the max name set by the latest call to SetMaxName(), then the required information is about the item Name. Note that Name must be a valid item name, i.e., a name that has been previously used in some call to SetGiName(), and not used in any call to Deleted() after the last call of SetGiName() where it was used. If Name == n, then the required information is about the constant subgradient of the linear 0-th component of Fi. Finally, if Name > n then the required information is about the newly produced "unnamed" item. Note that, given Name, it is immediately known to which component of Fi the associated item belongs.
GetGi() can be used to retrieve (part of) the linear GetNumVar()-vector which characterizes the item. SubG is a pointer to a vector of SgNum of appropriate length (see below) where the item have to be written. The indices strt and stp tell that what is required is the part of the vector corresponding to variables with "names" comprised between strt (included) and min{ stp , GetNumVar() } (excluded); the name of a variable is just its position in the vector Lmbd. Upon return, the required information has to be written in SubG; the "format" of SubG depends on what is returned in the read-only pointer SGBse. If SGBse == 0, then SubG[] is in "dense" format, i.e., SubG[ i ] is the entry of the subgradient relative to variable strt + i for i in 0 .. min( GetNumVar() , stp ) - strt. If SGBse != 0, then it must point to the vector of indices of nonzero entries of the subgradient (ordered in increasing sense and Inf< Index >()-terminated). That is, for i = strt ... stp - 1, the "real" SubG is
/ SubG[ j ] if exists j < k s.t. SGBse[ j ] = i
SubG[ i ] = | \ 0 otherwise.
where k, the number of nonzeroes of SubG[], is returned by GetGi(). If SGBse == 0, then GetGi() should return min{ stp , GetNumVar() } - strt, i.e., the length of the "dense" SubG[]. Note that, if SGBse != 0, the Indices in SGBse[] are in the range [strt, min{ stp , GetNumVar() }).
According to whether the item is a subgradient or a constraint (which is revealed by the return value of Fi()), GetMaxNZ( wFi ) and GetMaxCNZ( wFi ) [see above] provide an upper bound on the maximum number of nonzeroes in the item; let us denote that by MxNZ. Then, the length of the vector SubG is required to be at least
min( MxNZ , min( GetNumVar() , stp ) - strt ) ,
i.e., the minimum possible amount that is guaranteed to contain all the possible nonzeroes. Hence, a non-0 SGBDim must always be returned if MxNZ < min( GetNumVar() , stp ) - strt ), as in this case the vector SubG is not long enough to accommodate (the required part of) an item in "dense" format. Also, note that the memory for SGBase has to be provided by the FiOracle, but the NDO solver is not allowed to keep the pointer and must copy the vector, if it so requires, because the content may change at the subsequent call to any method of the FiOracle.
Allowing a NDO solver to query information about items "stored" in the FiOracle [see SetGiName() below] has different uses. For instance, the NDO solver may use variables generation techniques which try to solve a NDO problem by restricting it to a (small) "active set" of the variables, minimizing the function in that subspace and possibly revising the active set. Thus, at some point the NDO solver may not need some of the information related to an item, but this information may still be required later. A possibly more important use, however, is tied to the handling of changes in the function, which may occur in applications as those alluded to in the general notes. The implementation of the methods AddVariables() and ChgSbG() of NDOSolver [see NDOSlver.h] typically require calls to GetGi() to get information about some components of the items "stored" in the FiOracle. Note that these methods of class NDOSolver, in turn, can be called by the FiOracle itself, e.g. in the method GetFiStatus() [see below], so that the FiOracle may well be "prepared" when a call occurs simply because the caller was, ultimately, itself.
Implements FiOracle.
void GetInitialPoint | ( | LMRow | lmb | ) |
GetInitialPoint() writed into lmb[], which must be long at least GetNumVar(), the "standard starting point" of the function:
...
17 Smooth. [ Convex ]. Somma dei mezzi quadrati: x0= 1 1 1 1 1 1 .
18 AbsVal. [ Convex ]. Somma dei moduli: x0= 1 1 1 1 1 1 .. . ..
19 Lewis x0 = -10 [1, 1,]
In some cases, a Lower Bound on the minimal value of Fi is known; if Fi is a Lagrangian function, for instance, the objective function value c( x ) of any dual feasible solution x (s.t. A( x ) [<]= b and x \in X) gives a valid lower bound. This value can be useful for NDO solver, especially if it is tight; for one thing, it ensures that the function is not unbounded below.
GetLowerBound() allows to retrieve this value from the FiOracle, which may return - INF to signal that no Lower Bound is known. Note that the value may change—typically, increase—over time, e.g. as better feasible solutions of the dual problem are generated, so that it may be worth to call GetLowerBound() after each call to Fi() to check if a better bound has become available. Of course, the value returned by GetLowerBound() must be no smaller than that returned by GetMinusInfinity() [see above], although the two can be equal.
Furthermore, changes in Fi() (number of variables etc.) may possibly render any previously obtained Lower Bound invalid, so that this method should be called at least each time Fi() changes; failure to do that could lead to early and incorrect termination of the optimization due to the use of an incorrect Lower Bound.
Lower Bounds may be available separately for the wFi-th component of Fi, 1 <= wFi <= GetNrFi(); these are returned by GetLowerBound( wFi ). Note that if wFi is an "easy" component then it makes no sense to provide an explicit lower bound, since everything is known for that component and the Lower Bound, if any, is already implicit in the description; thus, GetLowerBound( wFi ) should always return - INF if wFi is "easy". GetLowerBound( 0 ) also makes no sense (i.e., - INF is again a good return value) while GetLowerBound( wFi ) for wFi > NrFi must return a "global" Lower Bound for the whole Fi.
Reimplemented from FiOracle.
|
inline |
Returns the name of the function (between 1 and 250.
GetVal() [and GetGi(), see above] can be used to query information about the items. The method allows to access both the newly produced item corresponding to the last call to NewGi() [see above], if it has not been "named" yet [see SetGiName() below], and all "named" items recorded in the FiOracle memory, if any [see SetMaxName() above]. This is specified by the value of Name. If Name < n, where n is the max name set by the latest call to SetMaxName(), then the required information is about the item Name. Note that Name must be a valid item name, i.e., a name that has been previously used in some call to SetGiName(), and not used in any call to Deleted() after the last call of SetGiName() where it was used. If Name == n, then the required information is about the constant subgradient of the linear 0-th component of Fi. Finally, if Name > n then the required information is about the newly produced "unnamed" item. Note that, given Name, it is immediately known to which component of Fi the associated item belongs.
The meaning of the value returned by GetVal( Name ) is different if Name indicates a subgradient or a linear constraint.
If Name indicates a constraint, then it is
SubG * Lambda <= GetVal( Name ),
i.e., GetVal() returns its right hand side.
If Name indicates an epsilon-subgradient, then GetVal() returns the value of the corresponding linearization of Fi in Lambda, that is, the smallest (known) value epsilon >= 0 such that the item Name is an epsilon-subgradient of the wFi-th component of Fi. In the Lagrangian case, if x is the dual solution corresponding to the epsilon-subgradient Name, it is
epsilon = Fi[ wFi ]() - ( c[ wFi ]( x ) - Lambda * A[ wFi ]( x ) )
i.e., how much "suboptimal" is the dual solution x for the wFi-th Lagrangian problem (x[ i ] is epsilon-optimal). A special case is when Name == n, i.e., the unique (sub)gradient of the linear 0-th component of Fi is queried; in this case, GetVal() returns the constant ‘b0’.
Allowing a NDO solver to query information about items "stored" in the FiOracle [see SetGiName() below] is fundamental for the handling of changes in the function, which may occur in applications as those alluded to in the general notes. The implementation of the method ChgFiV() of NDOSolver [see NDOSlver.h] typically requires calls to GetVal() to get the new Fi-values/right hand sides for subgradients/constraints "stored" in the FiOracle. Note that, in this case, the special return value - INF for GetVal( Name ) tells that the item ‘Name’ is no longer a valid one for the new function. Note that that method of class NDOSolver, in turn, can be called by the FiOracle itself, e.g. in the method GetFiStatus() [see below], so that the FiOracle may well be "prepared" when a call occurs simply because the caller was, ultimately, itself.
An implementation is given in the base class for those oracles which only return "exact" subgradients and no constraints, and do not store past solutions.
Reimplemented from FiOracle.