# H

C"t)min < U*(t) < (^)max if jj— = 0. (7.20)

Further details on the derivation of Eqs. 7.10 through 7.20 are discussed in Bryson and Ho . As long as H is a nonlinear function of uit Eq. 7.20 provides an explicit expression for u*(t).

7.2.3 Singular Control

If the Hamiltonian H varies linearly with Ui, i.e., if

with u' being the vector obtained from u by excluding uu then w,(i) cannot be obtained explicitly from the condition in Eq. 7.20 if hi is trivial over a finite time interval (ti < t < The control over each such finite interval is referred to as singular control and the time interval is referred to as singular control interval. The singular control problems are especially difficult to handle due to difficulties associated with identification of singular arc (trajectory of u*(t)), and estimation of when to transit from boundary control [u* = ('¡i,)rnjn or (wjViax] to singular control and vice versa. Triviality of hi over a finite time interval implies triviality of first and higher derivatives of hi with respect to time over the entire time interval. As will become evident in the illustrations presented later, this property is used to identify u* in a singular control interval. Admissibility of singular control is related to the kinetics of the process being optimized, i.e., the elements of f. If the bioprocess kinetics is such that singular control is not admissible, then the optimal control policy (trajectory of a manipulated input u%) would involve operation at the lower or upper bounds for ut [(u,)min or (Climax] or a composite of operations at the lower and upper bounds such that u*{t) = {ut)min if hi <0 (7.22a)

The trajectory of u*(t) then may involve one or more switches from the lower bound to upper bound and vice versa. The values of t at which such switches occur are called switching times.

### 7.2.4 Optimal Control

Next we consider situations where integral constraints in Eqs. 7.5 are applicable. Let there be a equality constraints and b inequality constraints. The original vector of state variables can be augmented by additional (a + b) state variables satisfying the following relations dx

= ^(x, u), Xj(0) = 0, j = (n + 1), (n + 2),..., (n + a) (7.23)

= ijij(x, u), Xj(0) = 0, j = (n+a+1), (n+a+2),..., (n+a+6) (7.24)

with x — [xi X2 ... xn\T being the original vector of n state variables. The vector of state variables may have to be further augmented if the objective function cannot be directly expressed in the form displayed in Eq. 7.3. Consider for example batch and/or fed-batch operation of a bioprocess. For cost-effective operation, it may be of interest to maximize productivity of the target product P. The objective function in this case would be

For a single-cycle operation, P(0) and V'(0) will be known (specified). The objective function above can be expressed as in Eq. 7.3 by augmenting the vector of state variables by an additional variable satisfying the following dx-j/dt = 1, Xj(0) = 0, j = {n + a + b+ 1). (7.26)

The objective function in Eq. 7.25 can now be expressed as in Eq. 7.3 with g(x, u) being trivial. The application of optimal control theory then follows as discussed before with the vector of state variables now consisting of n process variables and additional state variables satisfying relations such as Eqs. 7.23, 7.24 and 7.26. Since the Hamiltonian in Eq. 7.10 is independent of Xj (j = n + 1, n + 2,..., n + a + b), it follows from Eq. 7.11 that the corresponding adjoint variables, Xj (j = n + I,n + 2,... ,n + a + b), are time-invariant. For a steady-state operation, all adjoint variables Xj (j — 1,2,..., n + a + 6 + 1) are time-invariant and still provided by Eq. 7.11.

While integral constraints can be handled by augmenting the state variable space, if the process to be optimized is subject to algebraic constraints of the type

then the Hamiltonian in Eq. 7.10 must be appended to account for these constraints. The case study which follows provides an illustration of this.

7.2.5 Case Study - Feeding Policy in Single-Cycle and Repeated Fed-Batch Operations

Fed-batch operation is used for production of a variety of biochemicals [452, 642], Formation of the desired product(s) can be optimized by proper manipulation of the feed conditions (feed rate and feed composition). Here we consider a bioprocess represented by an unstructured model. Assuming that formation of biomass (X), utilization of the limiting substrate (5), and formation of the desired non-biomass product (P) represent the key rate processes and the biomass-specific rates of these can be expressed exclusively in terms of concentrations of cell mass, limiting substrate and the target non-biomass product, the dynamics of the bioreactor can be described by the following total and species material balances:

The bioreactor volume (V), volumetric feed rate (F), and the substrate feed concentration (Sf) are constrained as

Constraint 7.32 is necessary to prevent flooding. It is desired to maximize the objective function in Eq. 7.3 with g being considered constant (a special case) with x=[VXSPtf, the time variation of being described by Eq. 7.26 with n = 4 and a = b = 0. The Hamiltonian can be expressed as

where ho = (A2M - + X4e)X + X5 +g + rj0(V - Vm) (7.36)

r]o is a Lagrangian multiplier to take care of the algebraic inequality constraint, Eq. 7.32 (r)0(t) = 0 when V < Vm and rj0(t) > 0 when V = Vm) [392, 447, 454]. The variations in adjoint variables can be expressed as in Eq. 7.11, which assume the form

dX F

~ = -A2 - (m + Xfix)X2 + (a + Xax)X3 - (e + Xex)A4 (7.39) dX-t F

The subscripts X, S and P used in the above and elsewhere in the case studies in this chapter denote partial derivatives of a quantity (such as a specific rate or ratio of two specific rates) with respect to X, 5 and P, respectively. The Hamiltonian must be constant (= H*) on the optimal path, H* being zero when tj is not specified (Eq. 7.14).

The trajectory of x(i) will, in general, comprise an interior arc (V < Vm) and a boundary arc (V = Vm). When the orders of the boundary control and singular control are both unity, there is no jump in the adjoint variables at each junction point of the boundary and interior arcs [392, 447, 454], Once the bioreactor is full (V = Vrn), its operation continues on the boundary arc (F = 0) until t-tf .

It is evident from Eqs. 7.35 and 7.37 that the Hamiltonian is linear in both F and Sp. Admissibility of singular control must therefore be examined. The conditions for admissibility of singular control can be obtained from Eq. 7.20 as hi = h'i + h'2SF =0 in ti<t<t2 for dH/dF = 0 (7.43)

For the sake of illustration, we consider here u = F (i.e., Sp is not manipulated and kept time-invariant) and designate the control policy in singular control as Fs. The values of manipulated inputs in the singular control interval(s) are dependent on both the state and adjoint variables. For some specific cases of bioprocess kinetics, it is possible to express the optimal control policy, u, entirely in terms of state variables. These cases are considered here for the purpose of illustration. Such control policy can be easily implemented in a feedback mode.

Case 1. The three specific rates are related to one another by two linear relations as cr = p/j, and e — c/j, (7-45)

with p and c being constant. Substituting these relations and Eq. 7.28 into Eqs. 7.29-7.31, eliminating the specific cell growth rate between any two of these and integrating the resulting equations, it can be deduced that the bioreactor state moves along the intersection of the hyper planes

Satisfaction of relations in Eq. 7.45 implies that the bioreactor dynamics can be completely described by two algebraic relations, Eqs. 7.46 and 7.47 and two differential equations among Eqs. 7.28-7.31. Utilizing triviality of hi and dhi/dt, it can be deduced that the bioreactor trajectories must lie on the surface g(X, S, P) = Xfix - (SF - S)vs + P/xp = 0 (7.48)

during singular control intervals. The feed rate during the singular control interval is then obtained from triviality of d2hi/dt2 as

If Xq, So and P0 for a typical cycle lie on the line

p c then X, S and P lie on the same line during that cycle (Eqs. 7.46 and 7.47). The expressions for the singular surface and control policy during singular control, Eqs. 7.48 and 7.49, then reduce to

ab p p and

where Si is the substrate concentration at which conditions in Eq. 7.50 and 7.51 are satisfied. In this special situation, S, X and P remain timeinvariant at 5,, Xt and Pi, respectively (Si, Xt and Pi satisfy relations in Eq. 7.50). After substitution of relation Eq. 7.52 into total mass balance, Eq. 7.28, and integration of the same, one can deduce that both Fs and V vary exponentially with time.

In a strictly batch operation (F = 0), it can be deduced from Eqs. 7.297.31 and 7.45 that the concentration trajectories will lie on the line in Eq. 7.50. Further, in another related operation, viz., a continuous culture at steady state, X, S and P also lie on the line in Eq. 7.50.

In a typical cycle of a fed-batch operation, increase in V implies that the bioreactor state moves closer to the line defined by Eq. 7.50 if not already on it at the beginning of that cycle (see Eqs. 7.46 and 7.47). The feed point (S = Sf, X = P = 0) also lies on the line defined by Eq. 7.50. In a cyclic operation, the bioreactor contents are partially or completely withdrawn at the termination of a cycle; this is followed by addition of fresh feed. The initial state for the reactive portion of the next cycle, (Xln, Sin, Pin), therefore moves closer to the line defined by Eq. 7.50 if not already on it. It follows then that in a cyclic fed-batch operation with reproducible cycles, the concentration trajectories lie on the line in Eq. 7.50.

Relations 7.45 imply that among the three rate processes under consideration, only one (for example, cell growth) is independent. Optimal singular control interval therefore involves maximization of the specific cell growth rate, as indicated by Eqs. 7.48 and 7.51.

In general, ¡ix and fip are non-positive. Singular control is therefore feasible only when /¿s < 0 (Eqs. 7.48 and 7.51). Since fj, increases with increasing S at low values of the same, this then requires that /i exhibit non-monotonic behavior with respect to S. Singular control is therefore not admissible if fx is a monotonically increasing function of S. this being the case with some fermentations producing alcohols [47, 239, 325, 338, 544].

Case 2. The three specific rates are related to one another by a single linear relation as

with A, B and C being constants, at least two of which are non-zero. In view of Eq. 7.53, it can be deduced that, in a typical cycle, the bioreactor state lies on the hyperplane

[AX + B(Sf -S) + CP]V = c3 = [AXo + B(SF - S0) + CP0}V0 (7.54)

We consider here the special case where g is trivial and G is independent of tj in Eq. 7.3, and tj is not specified (therefore H = As = 0). It then follows that ho (Eqs. 7.35 and 7.36) must be trivial in a singular control interval. Two types of linear relations among the three rate processes under consideration are commonly encountered in bioprocess kinetics. The first type arises when cell growth and synthesis of the target non-biomass product account almost entirely for substrate utilization, i.e., when a = an + be (a = -A/B and b = -C/B, B=£ 0). (7.55)

In the other type, synthesis of the target non-biomass product is associated with and proportional to cell growth, with cell growth and substrate utilization being linearly independent rate processes, i.e., e = ctj, {c=-A/C,B = 0). (7.56)

When Eq. 7.55 is satisfied, the following necessary and sufficient condition for admissibility of singular control is obtained in view of the triviality of ho and dh\/dt in a singular control interval.

The above relation provides the description of the singular surface in the three dimensional concentration space (X, 5, P). The feeding policy during singular control is obtained from the triviality of cPhi/dt2 as

where

When Eq. 7.56 is applicable, the following necessary and sufficient condition for admissibility of singular control, which also describes the singular surface in the (X, S, P) plane, can be obtained in view of triviality of h0 and dh\/dt.

The feeding policy during the singular control is obtained as in Eq. 7.58a (since d2hi/dt2 is trivial), with a, 0 and 7 being defined as a - <?nx - 0 ~ ons - ^s, 7 = (r/J,p - n<rP. (7.60)

If Xq, So and Pq in a typical cycle lie on the plane

then X, S and P lie on the same during that cycle (see Eq. 7.54). Thebiore-actor trajectories can then be completely described in a two-dimensional phase-plane (S — X if B — 0 or X — P if B 0) with the singular arc (Xi, Si and Pi moving along the singular arc) being the intersection of the singular surface in Eq. 7.57 (if B is non-zero) or Eq. 7.59 (if B — 0) with the plane in Eq. 7.61.

The feed point (S = SF, X = P = 0) lies on the plane in Eq. 7.61. Further, in a strictly batch operation (F = 0), it can be deduced from Eqs. 7.29-7.31 and 7.53 that the concentration trajectories will lie on the plane in Eq. 7.61. Moreover, for a continuous culture at steady state, X, S and P also lie on the plane in Eq. 7.61. In a typical cycle of a fed-batch operation, increase in V implies that the bioreactor state (in terms of concentrations) moves closer to the plane in Eq. 7.61 if not already on it at the start of that cycle (see Eq. 7.54). In a cyclic fed-batch operation, the bioreactor contents are partially or completely withdrawn at the end of each cycle, which is followed by rapid addition of fresh feed. The initial state of the reactive portion of the next cycle, (Xq, So, Pq), therefore moves closer to the plane in Eq. 7.61 if not already on it. One can conclude then that in a repeated fed-batch operation with reproducible cycles, all concentration trajectories will lie on this plane. The trajectories in a batch operation in the two-dimensional phase-plane (S - X if B = 0 or X - P if B ^ 0) will in general be nonlinear. These can in some cases have inflection points. The locus of inflection points is described by one of the following surfaces :

/M-) - <M - I + M - ) =°> U a = an + be (7.62)

The intersections of surfaces in Eqs. 7.57 and 7.62 or those of surfaces in Eqs. 7.59 and 7.63, as appropriate, are of special significance in a fed-batch operation. At each such intersection,

These intersections are discrete points on the stoichiometric plane in Eq. 7.61. These points are referred to as limit points or singular inflection points. At a limit point, the feeding policy in Eqs. 7.58 and 7.60 reduces to

with Xi, Si and Pi satisfying Eq. 7.64 and being obtained from solutions of Eqs. 7.57 or 7.59, as appropriate, and Eq. 7.61. It follows from Eqs. 7.64 and 7.65 that Xi, Si and Pi are equilibrium (time-invariant) solutions of Eqs. 7.29-7.31. Upon reaching the limit point, Fs varies exponentially with time until transition to boundary control (F — 0) occurs upon saturation of the bioreactor volume (V = Vm). The bioreactor operation at the limit point is a quasi-steady state operation since X, S, and P remain timeinvariant while V increases with t.

In view of the nonlinear dependence of the three specific rates on X, S and P, multiplicity of limit points for a fixed feed composition cannot be ruled out. In a repeated fed-batch operation, the bioreactor trajectories during singular control interval of each cycle must terminate at a locally asymptotically stable (accessible) limit point. The necessary and sufficient conditions for local accessibility of a limit point can be obtained via linearized stability analysis of Eqs. 7.29-7.31. These conditions have been reported .

The first example of kinetics belonging to this case considered by Parule-kar  pertains to binary quasi-linear relations among /¿, a and e; ¡j = /¿(X, S, P), a = pfx + q, £ = c/x + e; c, e, p and q are constants (at least e or q is non-zero). This kinetics is applicable for bioprocesses where the limiting substrate is utilized for cell growth and maintenance and/or product synthesis, the synthesis of the target product being partially growth associated and partially non-growth associated. Eqs. 7.57 (e / 0) and 7.59 (e = 0) reduce in this case to condition in Eq. 7.48. In general, /ux and p,p are non-positive. Singular control is therefore feasible only when /¿s < 0 (Eqs. 7.48 and 7.51). Since fx increases with increasing S at low values of the same, this then requires that /i exhibit non-monotonic behavior with respect to S. Specific examples of this kinetics include (i) production of propionic acid by Propionibacterium shermanii  for which a — pfi, e = c/j. + e; and (ii) production of ammonium lactate by Lactobacillus species  for which cr = be, e = c^i + e, ¡1 being function of S and P in both cases. For both bioprocesses, ¡1\$ is positive and fip is negative for all S and P. Singular control is therefore inadmissible for either process.

The second example pertains to ethanol production from glucose by Saccharomyces cerevisiae [8, 9]. Two different forms of kinetics of cell growth, substrate utilization and product (ethanol) formation have been proposed for this bioprocess, these being

H = m(S)n2(P), e = ei(S)e2(P), cr = pu, p,x{S) = g^ »

The third example pertains to ethanol production from cellulose hydrolysate by S. cerevisiae. The following kinetic expressions have been used for description of this bioprocess [190, 600].

For these examples, optimization of single-cycle (once-through) and cyclic batch and fed-batch operations of these bioprocesses has been investigated in detail by Parulekar . Here, we import numerical illustrations for the second example.

The values assigned for the kinetic parameters in Eqs. 7.66 and 7.67 were [8, 9]

Kx = 0.22 g/L, K2 = 0.44 g/L, = 0.408 h~l, p = 10, em = 1.0 h~\ K3 = 16 g/L, Ka = 71.5 g/L, ai = 0.028 L/g, fix = 0.015 L/g. (7.69)

The performance index considered was the product (ethanol) yield at the end of the bioreactor operation, viz., J = P//5f, Pj = P(tf). The performance of the optimal fed-batch operation was compared for two types of cyclic batch operation. Each cycle of a cyclic batch operation consists of filling the reactor rapidly (Fm —> oo) with feed to increase the reactor volume from an initial volume Vq to Vm (0 < Vo < Vm), followed by a batch operation until the objective function J is maximized and then terminating the cycle by rapid withdrawal of the reactor contents to reduce the bioreactor volume from Vm to Vo- A batch operation is normally continued until the stoichiometrically limiting nutrient (limiting substrate here) is completely utilized and/or product synthesis is terminated, for this ensures that the bioreactor contents at the end of each cycle will have the maximum product concentration for a given feed composition. The batch reactor trajectories would terminate at (Xf, Sf, Pf) starting from (Xq. So, Po) with the feed point being (0, Sf, 0). The relations among the three concentration variables at these points are

The cyclic batch operations with Vo = 0 are referred to as operations without recycle (from one batch to the next) while those with non-zero Vq are termed as operations with recycle.

For the kinetics described in Eq. 7.66, a unique and locally asymptotically stable limit point on the singular axe is guaranteed for all Sp with the exception of very low Sf (Figure 7.1(a)). In a cyclic operation, the bioreactor trajectories in fed-batch mode terminate at a limit point and the trajectories approach the limit point in a single-cycle operation. The overall product-to-substrate yield for each of the three operations under consideration and the substrate and product concentrations at the limit point, Si and P,, respectively, all increased with increasing Sp (Figure 7.1). It is evident from Figure 7.1(b) that for the kinetics under consideration, the cyclic fed-batch operations are superior to cyclic batch operations with recycle, which in turn are superior to cyclic batch operations without recycle. The differential in the product yield between the optimal (fed-batch) operation and the two suboptimal (batch) operations increases as Sp is increased (Figure 7.1(b)). The maximum theoretical yield of ethanol based on glucose is 0.5111. For the kinetic parameters considered, Eq. 7.69, the overall ethanol yields for cyclic fed-batch operations exceed the maximum theoretical yield for Sp in excess of 76.6 g/L. The magnitudes of some of the kinetic parameters in this Sp range are therefore suspect.

Results for the kinetics described in Eqs. 7.66, 7.67, and 7.69 are presented in Figure 7.2. The singular control in this case has a richer variety. The number of limit points for the singular arc is (i) zero if Sf < 3.1957 g/L or if SF > 125.317 g/L, (ii) one if 3.1957 g/L < Sp < 8.0894 g/L, and (ii) two if 8.0894 g/L < SF < 125.317 g/L. The critical SF (125.317 g/L)  Figure 7.1. Profiles of (a) Si (---) and Pi (-), and (b) overall product yields for repeated batch operation without recycle (lower solid curve) and repeated fed-batch operation (upper solid curve) for fermentation described by Eq. 7.66. The dashed curve in (b) represents the upper (open) bound on the profiles of the overall product yield for repeated batch operations with recycle .

Figure 7.1. Profiles of (a) Si (---) and Pi (-), and (b) overall product yields for repeated batch operation without recycle (lower solid curve) and repeated fed-batch operation (upper solid curve) for fermentation described by Eq. 7.66. The dashed curve in (b) represents the upper (open) bound on the profiles of the overall product yield for repeated batch operations with recycle .

also represents the bifurcation point for the limit-point curve (Figure 7.2). Limit points lying on the lower branch (portion CDE) of the limit-point