Phase Space Representation of Dynamical Systems
The differential constraints defined in Section 13.1 are often called kinematic be- cause they can be expressed in terms of velocities on the C-space. This formulation is useful for many problems, such as modeling the possible directions of motions for a wheeled mobile robot. It does not, however, enable dynamics to be expressed. For example, suppose that the simple car is traveling quickly. Taking dynamics into account, it should not be able to instantaneously start and stop. For example, if it is heading straight for a wall at full speed, any reasonable model should not allow it to apply its brakes from only one millimeter away and expect it to avoid collision. Due to momentum, the required stopping distance depends on the speed. You may have learned this from a drivers education course.
To account for momentum and other aspects of dynamics, higher order differ- ential equations are needed. There are usually constraints on acceleration q¨, which is defined as dq˙/dt. For example, the car may only be able to decelerate at some maximum rate without skidding the wheels (or tumbling the vehicle). Most often, the actions are even expressed in terms of higher order derivatives. For example, the floor pedal of a car may directly set the acceleration. It may be reasonable
to consider the amount that the pedal is pressed as an action variable. In this case, the configuration must be obtained by two integrations. The first yields the velocity, and the second yields the configuration.
The models for dynamics therefore involve acceleration q¨ in addition to velocity q˙ and configuration q. Once again, both implicit and parametric models exist. For an implicit model, the constraints are expressed as
gi(q¨, q˙, q) = 0. (13.27)
For a parametric model, they are expressed as
q¨ = f(q˙, q, u). (13.28)
- Reducing Degree by Increasing Dimension
Taking into account constraints on higher order derivatives seems substantially more complicated. This section explains a convenient trick that converts con- straints that have higher order derivatives into a new set of constraints that has only first-order derivatives. This involves the introduction of a phase space, which has more dimensions than the original C-space. Thus, there is a trade-off because the dimension is increased; however, it is widely accepted that increasing the di- mension of the space is often easier than dealing with higher order derivatives. In general, the term state space will refer to either C-spaces or phase spaces derived from them.
- The scalar case
To make the discussion concrete, consider the following differential equation:
y¨ − 3y˙ + y = 0, (13.29)
in which y is a scalar variable, y ∈ R. This is a second-order differential equation because of y¨. A phase space can be defined as follows. Let x = (x1, x2) denote a two-dimensional phase vector, which is defined by assigning x1 = y and x2 = y˙. The terms state space and state vector will be used interchangeably with phase space and phase vector, respectively, in contexts in which the phase space is defined. Substituting the equations into (13.29) yields
y¨ − 3x2 + x1 = 0. (13.30)
So far, this does not seem to have helped. However, y¨ can be expressed as either x˙ 2 or x¨1. The first choice is better because it is a lower order derivative. Using x˙ 2 = y¨, the differential equation becomes
x˙ 2 − 3x2 + x1 = 0. (13.31)
Is this expression equivalent to (13.29)? By itself it is not. There is one more
constraint, x2 =
x˙ 1. In implicit form,
x˙ 1 − x2 = 0. The key to making the
phase space approach work correctly is to relate some of the phase variables by derivatives.
Using the phase space, we just converted the second-order differential equation (13.29) into two first-order differential equations,
x˙ 1 = x2
x˙ 2 = 3x2 − x1,
(13.32)
which are obtained by solving for x˙ 1 and x˙ 2. Note that (13.32) can be expressed as x˙ = f(x), in which f is a function that maps from R2 into R2.
The same approach can be used for any differential equation in implicit form,
g(y¨, y˙, y) = 0. Let x1 = y, x2 =
y˙, and
x˙ 2 =
y¨. This results in the implicit
equations g(x˙ 2, x2, x1) = 0 and x˙ 1 = x2. Now suppose that there is a scalar
action u U = R represented in the differential equations. Once again, the
∈
same approach applies. In implicit form, g(y¨, y˙, y, u) = 0 can be expressed as
g(x˙ 2, x2, x1, u) = 0.
Suppose that a given acceleration constraint is expressed in parametric form as y¨ = h(y˙, y, u). This often occurs in the dynamics models of Section 13.3. This can be converted into a phase transition equation or state transition equation of
the form x˙ = f(x, u), in which f : R2 × R → R2. The expression is
x˙ 1 = x2
x˙ 2 = h(x2, x1, u).
(13.33)
For a second-order differential equation, two initial conditions are usually given. The values of y(0) and y˙(0) are needed to determine the exact position y(t) for any t 0. Using the phase space representation, no higher order initial conditions
≥
are needed because any point in phase space indicates both y and y˙. Thus, given an initial point in the phase and u(t) for all t ≥ 0, y(t) can be determined.
Example 13.3 (Double Integrator) The double integrator is a simple yet im- portant example that nicely illustrates the phase space. Suppose that a second-
order differential equation is given as q¨ = u, in which q and u are chosen from R. In words, this means that the action directly specifies acceleration. Integrating5
once yields the velocity q˙ and performing a double integration yields the position
q. If q(0) and q˙(0) are given, and u(t′) is specified for all t′ [0, t), then q˙(t) and
∈
q(t) can be determined for any t > 0.
A two-dimensional phase space X = R2 is defined in which
x = (x1, x2) = (q, q˙). (13.34)
The state (or phase) transition equation x˙ = f(x, u) is
x˙ 1 = x2
x˙ 2 = u.
(13.35)
5Wherever integrals are performed, it will be assumed that the integrands are integrable.
To determine the state trajectory, initial values x1(0) = q0 (position) and x2(0) = q˙0 (velocity) must be given in addition to the action history. If u is constant, then the state trajectory is quadratic because it is obtained by two integrations of a
constant function. ．
- The vector case
The transformation to the phase space can be extended to differential equations in which there are time derivatives in more than one variable. Suppose that q represents a configuration, expressed using a coordinate neighborhood on a smooth n-dimensional manifold . Second-order constraints of the form g(q¨, q˙, q) = 0 or g(q¨, q˙, q, u) = 0 can be expressed as first-order constraints in a 2n-dimensional state space. Let x denote the 2n-dimensional phase vector. By extending the method that was applied to the scalar case, x is defined as x = (q, q˙). For each
C
integer i such that 1 ≤ i ≤ n, xi = qi. For each i such that n + 1 ≤ i ≤ 2n,
xi = q˙i−n. These substitutions can be made directly into an implicit constraint to reduce the order to one.
Suppose that a set of n differential equations is expressed in parametric form as
q¨ = h(q, q˙, u). In the phase space, there are 2n differential equations. The first n
correspond to the phase space definition x˙ i = xn+i, for each i such that 1 ≤ i ≤ n.
These hold because xn+i = q˙i and x˙ i is the time derivative of q˙i for i n. The remaining n components of x˙ = f(x, u) follow directly from h by substituting the first n components of x in the place of q and the remaining n in the place of q˙ in the expression h(q, q˙, u). The result can be denoted as h(x, u) (obtained directly from h(q, q˙, u)). This yields the final n equations as x˙ i = hi−n(x, u), for each i
≤
such that n + 1 ≤ i ≤ 2n. These 2n equations define a phase (or state) transition
equation of the form x˙ = f(x, u). Now it is clear that constraints on acceleration
can be manipulated into velocity constraints on the phase space. This enables the tangent space concepts from Section 8.3 to express constraints that involve
acceleration. Furthermore, the state space X is the tangent bundle (defined in (8.9) for Rn and later in (15.67) for any smooth manifold) of C because q and q˙ together indicate a tangent space Tq (C) and a particular tangent vector q˙ ∈ Tq (C).
- Higher order differential constraints
The phase space idea can even be applied to differential equations with order higher than two. For example, a constraint may involve the time derivative of acceleration q(3), which is often called jerk. If the differential equations involve jerk variables, then a 3n-dimensional phase space can be defined to obtain first-order constraints. In this case, each qi, q˙i, and q¨i in a constraint such as g(q(3), q¨, q˙, q, u) = 0 is defined as a phase variable. Similarly, kth-order differential constraints can be reduced to first-order constraints by introducing a kn-dimensional phase space.
Example 13.4 (Chain of Integrators) A simple example of higher order dif- ferential constraints is the chain of integrators.6 This is a higher order generaliza- tion of Example 13.3. Suppose that a kth-order differential equation is given as q(k) = u, in which q and u are scalars, and q(k) denotes the kth derivative of q with respect to time.
A k-dimensional phase space X is defined in which
x = (q, q˙, q¨, q(3), . . . , q(k−1)). (13.36)
The state (or phase) transition equation x˙ = f(x, u) is x˙ i = xi+1 for each i such
that 1 i n 1, and equivalent to q(k) = u.
≤ ≤ −
x˙ n = u. Together, these n individual equations are
The initial state specifies the initial position and all time derivatives up to order k 1. Using these and the action u, the state trajectory can be obtained by
−
a chain of integrations. ．
You might be wondering whether derivatives can be eliminated completely by introducing a phase space that has high enough dimension. This does actually work. For example, if there are second-order constraints, then a 3n-dimensional phase space can be introduced in which x = (q, q˙, q¨). This enables constraints such as g(q, q˙, q¨) = 0 to appear as g(x) = 0. The trouble with using such formulations is that the state must follow the constraint surface in a way that is similar to traversing the solution set of a closed kinematic chain, as considered in Section
4.4. This is why tangent spaces arose in that context. In either case, the set of allowable velocities becomes constrained at every point in the space.
Problems defined using phase spaces typically have an interesting property known as drift. This means that for some x X, there does not exist any u U such that f(x, u) = 0. For the examples in Section 13.1.2, such an action always existed. These were examples of driftless systems. This was possible because the constraints did not involve dynamics. In a dynamical system, it is impossible to instantaneously stop due to momentum, which is a form of drift. For example, a car will “drift” into a brick wall if it is 3 meters way and traveling 100 km/hr in the direction of the wall. There exists no action (e.g., stepping firmly on the brakes) that could instantaneously stop the car. In general, there is no way to instantaneously stop in the phase space.
∈ ∈
- Linear Systems
Now that the phase space has been defined as a special kind of state space that can handle dynamics, it is convenient to classify the kinds of differential models that can be defined based on their mathematical form. The class of linear systems has been most widely studied, particularly in the context of control theory. The
6It is called this because in block diagram representations of systems it is depicted as a chain of integrator blocks.
reason is that many powerful techniques from linear algebra can be applied to yield good control laws [192]. The ideas can also be generalized to linear systems that involve optimality criteria [28, 570], nature [95, 564], or multiple players [59]. Let X = Rn be a phase space, and let U = Rm be an action space for m n.
≤
A linear system is a differential model for which the state transition equation can
be expressed as
x˙ = f(x, u) = Ax + Bu, (13.37) in which A and B are constant, real-valued matrices of dimensions n × n and
n × m, respectively.
Example 13.5 (Linear System Example) For a simple example of (13.37), suppose X = R3, U = R2, and let
+
0 1
1
. (13.38)
2 0 1 x 3
1 1
x˙ 1
x˙
x˙ 2
3
0 √2 1 x1
=
1 −1 4
x2
1 0 （u ＼
u2
3
u2
Performing the matrix multiplications reveals that all three equations are linear in the state and action variables. Compare this to the discrete-time linear Gaussian
system shown in Example 11.25. ．
Recall from Section 13.1.1 that k linear constraints restrict the velocity to an (n k)-dimensional hyperplane. The linear model in (13.37) is in parametric form, which means that each action variable may allow an independent degree of
−
freedom. In this case, m = n − k. In the extreme case of m = 0, there are no
actions, which results in x˙ = Ax. The phase velocity x˙ is fixed for every point
x X. If m = 1, then at every x X a one-dimensional set of velocities may be chosen using u. This implies that the direction is fixed, but the magnitude is chosen using u. In general, the set of allowable velocities at a point x Rn is an
∈
∈ ∈
m-dimensional linear subspace of the tangent space Tx(Rn) (if B is nonsingular).
In spite of (13.37), it may still be possible to reach all of the state space from
any initial state. It may be costly, however, to reach a nearby point because of the restriction on the tangent space; it is impossible to command a velocity in some directions. For the case of nonlinear systems, it is sometimes possible to quickly reach any point in a small neighborhood of a state, while remaining in a small region around the state. Such issues fall under the general topic of controllability, which will be covered in Sections 15.1.3 and 15.4.3.
Although not covered here, the observability of the system is an important topic in control [192, 478]. In terms of the I-space concepts of Chapter 11, this means that a sensor of the form y = h(x) is defined, and the task is to determine the current state, given the history I-state. If the system is observable, this means that the nondeterministic I-state is a single point. Otherwise, the system may only be partially observable. In the case of linear systems, if the sensing model is also linear,
y = h(x) = Cy, (13.39)
then simple matrix conditions can be used to determine whether the system is observable [192]. Nonlinear observability theory also exists [478].
As in the case of discrete planning problems, it is possible to define differential models that depend on time. In the discrete case, this involves a dependency on stages. For the continuous-stage case, a time-varying linear system is defined as
x˙ = f(x(t), u(t), t) = A(t)x(t) + B(t)u(t). (13.40)
In this case, the matrix entries are allowed to be functions of time. Many powerful control techniques can be easily adapted to this case, but it will not be considered here because most planning problems are time-invariant (or stationary).
- Nonlinear Systems
Although many powerful control laws can be developed for linear systems, the vast majority of systems that occur in the physical world fail to be linear. Any differential models that do not fit (13.37) or (13.40) are called nonlinear systems. All of the models given in Section 13.1.2 are nonlinear systems for the special case in which X = .
C
One important family of nonlinear systems actually appears to be linear in some sense. Let X be a smooth n-dimensional manifold, and let U Rm. Let U = Rm for some m n. Using a coordinate neighborhood, a nonlinear system of the form
≤
⊆
m
-
x˙ = f(x) + gi(x)ui (13.41)
i=1
for smooth functions f and gi is called a control-affine system or affine-in-control system.7 These have been studied extensively in nonlinear control theory [478, 846]. They are linear in the actions but nonlinear with respect to the state. See Section 15.4.1 for further reading on control-affine systems.
For a control-affine system it is not necessarily possible to obtain zero velocity
because f causes drift. The important special case of a driftless control-affine system occurs if f ≡ 0. This is written as
m
-
x˙ = gi(x)ui. (13.42)
i=1
By setting ui = 0 for each i from 1 to m, zero velocity, x˙ = 0, is obtained.
Example 13.6 (Nonholonomic Integrator) One of the simplest examples of a driftless control-affine system is the nonholonomic integrator introduced in control literature by Brockett in [142]. It some times referred to as Brockett’s system, or the Heisenberg system because it arises in quantum mechanics [112]. Let X =
7Be careful not to confuse control-affine systems with affine control systems, which are of the form x˙ = Ax + Bu + w, for some constant matrices A, B and a constant vector w.
R3, and let the set of actions U = R2. The state transition equation for the nonholonomic integrator is
x˙ 1 = u1 x˙ 2 = u2
x˙ 3 = x1u2 − x2u1.
(13.43)
．
Many nonlinear systems can be expressed implicitly using Pfaffian constraints, which appeared in Section 13.1.1, and can be generalized from C-spaces to phase spaces. In terms of X, a Pfaffian constraint is expressed as
g1(x)x˙ 1 + g2(x)x˙ 2 + · · · + gn(x)x˙ n = 0. (13.44)
Even though the equation is linear in x˙ , a nonlinear dependency on x is allowed.
Both holonomic and nonholonomic models may exist for phase spaces, just as in the case of C-spaces in Section 13.1.3. The Frobenius Theorem, which is covered in Section 15.4.2, can be used to determine whether control-affine systems are completely integrable.
- Extending Models by Adding Integrators
The differential models from Section 13.1 may seem unrealistic in many applica- tions because actions are required to undergo instantaneous changes. For example, in the simple car, the steering angle and speed may be instantaneously changed to any value. This implies that the car is capable of instantaneous acceleration changes. This may be a reasonable approximation if the car is moving slowly (for example, to analyze parallel-parking maneuvers). The model is ridiculous, however, at high speeds.
Suppose a state transition equation of the form x˙ = f(x, u) is given in which
the dimension of X is n. The model can be enhanced as follows:
- Select an action variable ui.
- Rename the action variable as a new state variable, xn+1 = ui.
- Define a new action variable u′i that takes the place of ui.
- Extend the state transition equation by one dimension by introducing x˙ n+1 =
u′i.
This enhancement will be referred to as placing an integrator in front of ui. This procedure can be applied incrementally as many times as desired, to create a chain of integrators from any action variable. It can also be applied to different action variables.
- Better unicycle models
Improvements to the models in Section 13.1 can be made by placing integrators in front of action variables. For example, consider the unicycle model (13.18). Instead of directly setting the speed using us, suppose that the speed is obtained by integration of an action ua that represents acceleration. The equation s˙ = ua is used instead of s = us, which means that the action sets the change in speed. If ua is chosen from some bounded interval, then the speed is a continuous function of time.
How should the transition equation be represented in this case? The set of possible values for ua imposes a second-order constraint on x and y because double integration is needed to determine their values. By applying the phase space idea, s can be considered as a phase variable. This results in a four-dimensional phase space, in which each state is (x, y, θ, s). The state (or phase) transition equation is
x˙ = s cos θ y˙ = s sin θ
θ˙ = uω
s˙ = ua, (13.45)
which should be compared to (13.18). The action us was replaced by s because now speed is a phase variable, and an extra equation was added to reflect the connection between speed and acceleration.
The integrator idea can be applied again to make the unicycle orientations a continuous function of time. Let uα denote an angular acceleration action. Let ω denote the angular velocity, which is introduced as a new state variable. This results in a five-dimensional phase space and a model called the second-order unicycle:
x˙ = s cos θ s˙ = ua
y˙ = s sin θ θ˙ = ω,
ω˙ = uα (13.46)
in which u = (ua, uα) is a two-dimensional action vector. In some contexts, s may be fixed at a constant value, which implies that ua is fixed to ua = 0.
- A continuous-steering car
As another example, consider the simple car. As formulated in (13.15), the steering angle is allowed to change discontinuously. For simplicity, suppose that the speed is fixed at s = 1. To make the steering angle vary continuously over time, let uω
be an action that represents the velocity of the steering angle: φ˙ = uω . The result
is a four-dimensional state space, in which each state is represented as (x, y, θ, φ).
This yields a continuous-steering car,
x˙ = cos θ y˙ = sin θ
θ˙ = tan φ
L
φ˙ = uω , (13.47)
in which there are two action variables, us and uω . This model was used for planning in [849].
A second integrator can be applied to make the steering angle a C1 smooth function of time. Let ω be a state variable, and let uα denote the angular acceler- ation of the steering angle. In this case, the state vector is (x, y, θ, φ, ω), and the state transition equation is
x˙ = cos θ
φ˙ = ω
y˙ = sin θ
θ˙ = tan φ.
L
ω˙ = uα (13.48)
Integrators can be applied any number of times to make any variables as smooth as desired. Furthermore, the rate of change in each case can be bounded due to limits on the phase variables and on the action set.
- Smooth differential drive
A second-order differential drive model can be made by defining actions ul and ur that accelerate the motors, instead of directly setting their velocities. Let ωl and ωr denote the left and right motor angular velocities, respectively. The resulting state transition equation is
r
x˙ = (ωl + ωr) cos θ
2
r
y˙ = (ωl + ωr) sin θ
2
ω˙ l = ul
ω˙ r = ur (13.49)
θ˙ = r (ω − ω ).
L
r
l
In summary, an important technique for making existing models somewhat more realistic is to insert one or more integrators in front of any action variables. The dimension of the phase space increases with the introduction of each integra- tor. A single integrator forces an original action to become continuous over time. If the new action is bounded, then the rate of change of the original action is bounded in places where it is differentiable (it is Lipschitz in general, as expressed in (8.16)). Using a double integrator, the original action is forced to be C1 smooth. Chaining more integrators on an action variable further constrains its values. In general, k integrators can be chained in front of an original action to force it to be Ck−1 smooth and respect Lipschitz bounds.
One important limitation, however, is that to make realistic models, other variables may depend on the new phase variables. For example, if the simple car is traveling fast, then we should not be able to turn as sharply as in the case of a slow-moving car (think about how sharply you can turn the wheel while parallel parking in comparison to driving on the highway). The development of better differential models ultimately requires careful consideration of mechanics. This provides motivation for Sections 13.3 and 13.4.