Iterative feedback tuning of wind turbine controllers

Traditionally, wind turbine controllers are designed using first principles or linearized or identified models. The aim of this paper is to show that with an automated, online, and model-free tuning strategy, wind turbine control performance can be significantly increased. For this purpose, iterative feedback tuning (IFT) is applied to two different turbine controllers: drivetrain damping and collective pitch control. The results, obtained by high-fidelity simulations using the NREL 5MW wind turbine, indicate significant performance improvements over baseline controllers, which were designed using classical loop-shaping techniques. It is concluded that iterative feedback tuning of turbine controllers has the potential to become a valuable tool to improve wind turbine performance.


Introduction
The control system plays a crucial role in the operation of wind turbines (van Kuik et al., 2016).Without properly tuned control loops, the turbine does not extract the maximum amount of energy from the wind and loads might not be mitigated optimally.Typically, wind turbine controllers are designed using linearized models obtained from wind turbine software packages (Bossanyi and Witcher, 2009).The linearized models approximate the nonlinear wind turbine dynamics in the vicinity of selected operating points for which the controller is designed.To obtain a controller that performs satisfactorily across the different operating conditions, gain-scheduling techniques are often used.When necessary, the controllers can be fine tuned by connecting them to the nonlinear wind turbine model using high-fidelity software packages.
Several factors detriment the controller performance when implemented on the actual turbine.First, the controller is designed upon the basis of models.This directly implies that there will be modeling errors and, hence, differences with the actual turbine.Second, every turbine will be different from the specifications due to, for instance, manufacturing errors and imperfections (van der Veen et al., 2013a).Third, due to environmental differences such as varying soil dynamics throughout a wind plant, the dynamics of wind turbines vary per turbine (Lombardi et al., 2013;Abhinav and Saha, 2015).Finally, due to wear and tear, dynamics will change over time.All these factors impact the (controller) performance of wind turbines and should be addressed during commissioning and periodically during the lifetime of a wind turbine.
A wind turbine manufacturer has several opportunities to overcome the aforementioned issues.One of these is by applying system identification techniques (Hjalmarsson, 2005;van der Veen et al., 2013a, b;La Cava et al., 2016).With system identification, the dynamics of a wind turbine subsystem are obtained by exciting a wind turbine input and measuring the response thereof, which often yields a more realistic model.By using the identified models as a basis for the control design, performance can be increased.The drawback of this approach is that it might be time consuming to obtain the dynamics for all operating conditions, after which the controller needs to be redesigned.
In the past decade, research has been conducted on developing load-reducing controllers that "learn" the optimal controller settings online (van Wingerden et al., 2011;Houtzager et al., 2013;Navalkar et al., 2014Navalkar et al., , 2015;;Xiao et al., 2016).These controllers are typically scheduled on basis functions and thereby mainly target the periodic wind turbine loading.By minimizing a cost function with respect to the con-troller parameters, the optimal (linear) combination of the basis functions can be obtained online.The controllers have been successfully demonstrated in both simulation studies as well as experimental wind tunnel testing.However, the main drawback of the majority of the approaches is that the resulting controller operates in feedforward.This means that mainly the deterministic loads are targeted, while stochastic loads remain roughly unaffected.
Another strategy is to tune the controllers offline by using the previously mentioned models and use this as a starting point for an automated online tuning algorithm (which can be run during commissioning and periodically over the turbine lifetime for instance).One such algorithm is given by iterative feedback tuning (IFT) (Hjalmarsson et al., 1998;Hjalmarsson, 2002).With IFT, the parameters of a fixedstructure controller are iteratively optimized by carrying out two or more experiments on the closed-loop system, with which estimates of the gradient with respect to the controller parameters are obtained.Iteratively updating the controller parameters by using the gradient estimates then minimizes a user-defined cost function.The key advantage of IFT is that detailed knowledge of the (wind turbine) system is not needed, while the main requirement is that the initial closedloop system is stable, implying that it can be directly used for optimization of wind turbine controllers.IFT has been successfully applied to various application fields, including mechatronics (Al Mamun et al., 2007;Heertjes et al., 2016), robotics (Liu et al., 2011), process industry (Lequin, 1997), and recently wind turbines (Navalkar and van Wingerden, 2015).
In this paper, IFT is used to optimize the performance of an active drivetrain damper and of reference tracking using collective pitch control (CPC).The main contribution is therefore to show the (practical) application of IFT to existing wind turbines.This paper also contributes in showing how IFT can be applied to systems that have multiple controllers in the loop.Moreover, it is shown that IFT can be applied to systems that have a reference input with a static offset by performing an additional experiment.Finally, the impact of several practical considerations is shown, including the experiment length, signal-to-noise ratio, and convergence speed.
This paper is organized as follows.In the next section, the details of the IFT algorithm and the closed-loop experiments that are required to obtain gradient estimates are given.Subsequently, in Sect.3, short descriptions of the NREL 5MW wind turbine and the high-fidelity simulation environment are given.An overview of the control system is also provided.In Sects. 4 and 5, the IFT algorithm is used to optimize the performance of an active drivetrain damper and the reference tracking of the rotor speed using CPC.Finally, conclusions are drawn in Sect.6.

Iterative feedback tuning theory
This section introduces the IFT method.First, the main details of IFT are given in Sect.2.1, followed by the analysis of IFT for systems with a reference input containing an offset in Sect.2.3.

IFT introduction
The basic rationale of IFT (Hjalmarsson et al., 1998;Hjalmarsson, 2002) is to minimize a cost function, given for instance by the quadratic expression in an iterative manner.The cost function J (ρ) in Eq. ( 1) depends on the (tunable) controller parameters ρ, the squared error between the output y(k, ρ) and the reference input r(k) (possibly prefiltered with a system reference model), and the squared input signal of the system u(k, ρ), where k indicates the time instance.The cost function is divided by 2 times the number of data samples N , and involves the expectation E[•] due to noise.Using a gradient search of the type where i is the iteration number, ∂J /∂ρ(ρ i ) the gradient of the cost function Eq. ( 1), R i a positive definite matrix (e.g., the Hessian of Eq. 1), an unconstrained optimization problem is obtained.Note that the objective function can be non-convex, and will in such a case converge to a local minimum.
It is clear that minimizing Eq. ( 1) boils down to computing the gradient ∂J /∂ρ(ρ i ) and Hessian R i at every iteration.Previous studies (e.g., refer to Hjalmarsson et al., 1998;Hjalmarsson, 2002) have shown that these quantities can be obtained from the closed-loop system by conducting a number of experiments.To see this, first consider the partial derivative of the cost function J (ρ) with respect to the controller parameter vector ρ: which involves the signals ∂y/∂ρ(k, ρ) and ∂u/∂ρ(k, ρ).Thus, in each iteration the following signals are required: 1. the signals r(k), y(k, ρ i ), and u(k, ρ i ); 2. the gradients ∂y/∂ρ(k, ρ i ) and ∂u/∂ρ(k, ρ i ); 3. unbiased estimates of the products (y(k, ρ i ) − r(k))∂y/∂ρ(k, ρ i ) and u(k, ρ i )∂u/∂ρ(k, ρ i ).The signals of the first requirement can be obtained from running a closed-loop experiment as in Fig. 1.
Obtaining the signals of the latter two requirements is slightly more involved.In order to derive the required gradients and unbiased estimates, consider the block scheme of the closed-loop system in Fig. 1.From the block scheme, it is readily observed that where P is the plant, C(ρ) the fixed-structure parameterized controller, and v 1 the measurement noise.Taking the partial derivatives of the latter signals with respect to the controller parameters ρ gives Substitution of Eq. ( 7) in Eq. ( 6) yields where S(ρ) = (I + P C(ρ)) −1 is the sensitivity function.The gradient in Eq. ( 8) can be obtained by injecting r 1 (k) − y 1 (k, ρ) at the process input u 2 (k, ρ) according to Fig. 2.This experiment is the so-called gradient experiment.Notice that Fig. 2 includes a scaled injection signal with a factor F at the process input as well as a factor 1/F at the output, which will become clear in the following paragraphs; for now assume F = 1.It should also be noted that the gradient ∂y/∂ρ(k, ρ) obtained through Fig. 2 is an estimate of Eq. ( 8) because it is contaminated with noise v 2 (k), i.e., Finally, note that the subscript indicates the experiment number and should not be confused with the iteration number i. Performing the gradient experiment as in Fig. 2 avoids the need of an inverse of the controller, which will become clear in the next section.
The input gradient signal ∂u/∂ρ(k, ρ) can be obtained in a similar way (refer to Hjalmarsson et al., 1998, for a derivation).It can be derived that which means that the gradient can be obtained from Fig. 2 by multiplying the process input u 2 (k, ρ) with ∂C/∂ρ (or by 1/F • ∂C/∂ρ when the scaling factor F is applied).Again, this gradient is an estimate because it is contaminated with v 2 (k) and is therefore denoted by ∂u/∂ρ(k, ρ i ).
The matrix R in Eq. ( 2) is often replaced by an approximation of the Hessian.When y(k, ρ) − r(k) is small, the Gauss-Newton direction can be a suitable choice (Hjalmarsson, 2002).However, the approximated Hessian in Eq. ( 11) will be biased because of the disturbances.Typically, this will slow down the convergence of the algorithm.In Solari and Gevers (2004), it is shown that an unbiased estimate of the Hessian can be obtained on the basis of two additional closed-loop experiments.In this paper, however, it was found that the approximated Hessian Eq. ( 11) performed sufficiently well and was therefore the preferred choice.

Improving the signal-to-noise ratio
In order to improve the signal-to-noise ratio in the gradient experiment, it is suggested to scale the injection signal with a factor F (Hjalmarsson, 2002) as shown in Fig. 2. Consequently, the output ∂y/∂ρ(k, ρ) should be divided by the same scaling factor.For a discussion on the optimal choice of F , refer to Hjalmarsson and Gevers (1997).In the later sections, the gain F needs to not only be used to improve the signal-to-noise ratio but also to appropriately scale the input injection signal r 1 (k) − y 1 (k, ρ).In certain cases, the reference r(k) and/or measured signals y(k, ρ) can be orders of magnitude larger (or smaller) than the process input u(k, ρ), which means that injection of r 1 (k) − y 1 (k, ρ) with large amplitude would lead to an undesired input.Hence, F can be used to scale this signal to the desired input level.

IFT for systems with offset in reference inputs
In the previous subsection, the controller C was subject to IFT, where it was assumed that r(k) could be set to zero.In this subsection, the details of iteratively optimizing controller C when operating at a non-zero reference input r(k) are analyzed.Specifically, the details in this part are tailored to the case of optimizing the CPC, for instance.In the CPC loop, the generator speed is held close to the rated generator speed.Thus, the reference input can be used to generate a step response.Then, the reference r(k) can be written as r(k) = r o + r n (k), where r o = 0 is the constant offset (e.g., the rated generator speed) and r n the reference step.Similar to the previous section, the offset r o will contaminate the gradient signals and therefore a third experiment is required to remove this contamination.
The experiments to obtain unbiased gradient estimates are as follows (Hjalmarsson, 2002).First, an experiment is carried out where a step change r 1 (k) = r o + r n (k) is applied to the closed-loop system according to Fig. 1.Recording the process input u 1 (k, ρ) and the output y 1 (k, ρ), gives the first set of signals.Second, an experiment identical to the first experiment is carried out, except that there is not a step change in the reference; hence, r 2 (k) = r 0 .This yields the second set of signals: The third experiment is the experiment where the recorded signals of the first and second experiments are used to obtain an estimate of the gradient.The control configuration for the gradient experiment is shown in Fig. 3, where it is seen that the reference input equals r o .For this configuration, the gradient estimate signals read (also see Appendix A) from which it can be seen that both noise and the reference offset r o perturb the estimate.Fortunately, the offset r o in the gradient estimates can be removed by using Eqs.( 14) and ( 15) during the gradient experiment (see Fig. 3).
It is noted that the first and second experiments are nearly identical, but they are both required in order for (y 1 (k, ρ i ) − r 1 (k)) ∂y/∂ρ(k, ρ i ) and u 1 (k, ρ i ) ∂u/∂ρ(k, ρ i ) to be unbiased (Hjalmarsson, 2002) under the same assumptions as before.

IFT for systems with multiple controllers
The IFT method discussed in the previous part cannot directly be applied to control systems where multiple (decoupled) controllers are working on the same input signal.Consider in this case, for example, the torque controller and the drivetrain controller.The torque controller regulates the generator torque in such a way that the rotor speed provides maximum power extraction from the wind.At the same time, a drivetrain damper adds a small torque ripple on the regulated generator torque in order to reduce drivetrain oscillations (which will be further explained in Sect.4).Thus, in such a case the reference signal r(k) in Fig. 2 cannot be set to zero.Moreover, the torque and damping controllers might have some interaction such that biased estimates are obtained.In order to apply IFT to this kind of control system, it is shown that a similar procedure as for the case with reference offsets can be used.
The basic control system analyzed in this section is shown in Fig. 4. The control system consists of controllers C 1 and C 2 (ρ), where the latter is subject to optimization.Controller C 1 works on the error between a reference r(k) and the measured output y(k, ρ), while C 2 (ρ) works on the output y(k, ρ) filtered by H(s).The role of the high-pass filter will become clear at a later point.The controller C 2 (ρ) is implemented with positive feedback without loss of generality.
The first experiment is identical to the single controller case as described in the previous paragraphs: the closed-loop system in Fig. 4 is used to obtain the following signals: where S = (I + P C 1 − P C 2 (ρ)H) −1 is the sensitivity function.The signals in Eqs. ( 18)-( 19) determine the cost function given in Eq. ( 1).
The gradient signal related to the output y 1 (k, ρ) is obtained as follows.First note that such that the partial derivative thereof is given by Then, note that the process input u(k, ρ) in Fig. 4, has partial derivative Substituting Eq. ( 23) in Eq. ( 21) and manipulating it yields Similarly, the gradient related to the input can be found to be (see Appendix B) Thus, the signals in Eqs. ( 24)-( 25) can be obtained by injecting the high-pass filtered output Hy(k, ρ) or simply y (k, ρ) at the process input u(k, ρ) in a gradient experiment according to Fig. 5. Doing so, one then obtains the following gradient estimates: Compare the equations above with Eqs. ( 24)-( 25) and it can be observed that the estimates are biased.Now, by running an experiment identical to the first experiment, one acquires y 2 (k, ρ) and u 2 (k, ρ) identical to Eqs. ( 18)-( 19).Subtracting these, as shown in Fig. 5, cancels the undesired terms in the gradient signals in Eqs. ( 26)-( 27) such that the desired gradient is obtained, which is only perturbed by noise terms.The high-pass filter H is incorporated for practical consideration.In the gradient experiment, the signal F Hy 1 (k, ρ) is injected into the system.In the case of drivetrain damping, this would mean injecting the measured generator speed y 1 (k, ρ) with a scaling factor F .As the generator speed during operating is larger than zero, this would imply inserting a step change in the demanded generator torque.Highpass filtering the measured speed causes the injection of a signal that varies around zero.

Including cost function weights
This section shortly discusses weighting filters in the cost function.The cost function in Eq. ( 1) can be modified to include time and frequency weights.First, consider the following so-called zero-weighting mask of the cost function (Lequin et al., 1999(Lequin et al., , 2003) www.wind-energ-sci.net/2/153/2017/Wind Energ.Sci., 2, 153-173, 2017 where the index of the summation starts at N 0 > 1.The motivation for this mask, often used for step response tuning, is as follows.Typically, the main objective of step response tuning is to move the system quickly from one point to another.The settling time is an important parameter in this context.Without zero-weighting mask, the controller is tuned such that the reference is as closely matched as possible, while in practice one does not care too much about the trajectory to the new reference value, as long as the overshoot is not too large.By zero weighting the transient trajectory in the cost function, the algorithm tries to achieve a fast settling time.
The cost function can also incorporate frequency weights L y and L u which filter the error y(k, ρ) − r(k) and input u(k) accordingly.With the frequency weights, one can emphasize or suppress frequency bands in the cost function.For example, in the case of drivetrain damping, the measured output signal will not only be composed of the drivetrain resonance frequency but also of many other disturbances with various frequency components.Thus, filtering the drivetrain frequencies in the cost function makes sure the controller will focus on the filtered frequencies.
The cost function Eq. ( 29) requires filtering of y(k, ρ) − r(k) and u(k, ρ) with L y and L u , respectively.Moreover, the derivative of Eq. ( 29) with respect to the control parameters then involves the filters L y and L u as well.Thus, this also requires the gradient signals to be passed through the frequency filters.

Stability and convergence
The iterative optimization of the controller parameters ρ described in the previous paragraphs can have robustness issues.This is caused by the fact that, typically, there are no guarantees that the controller remains stable during the iteration.For that reason a number of articles (Heertjes et al., 2016;Procházka et al., 2005;de Bruyne and Kammer, 1999;Veres and Hjalmarsson, 2002) have appeared on the subject of including stability constraints for the optimization procedure.These algorithms are not considered here because no stability issues are encountered in this study.However, a safety measure that can be taken is lowering the parameter update step size γ i , but for this a trade-off between stability and convergence rate has to be made.A technique to improve convergence is to adjust the step size γ i after each iteration, based on a norm of the cost function gradient ∂J /∂ρ(ρ i ).

Wind turbine and simulation environment
In this study, the NREL 5MW reference wind turbine model is used.The model does not represent an existing wind turbine but is considered to reflect typical commercial wind turbines of similar ratings.The turbine has three blades in an upwind rotor configuration, a rotor diameter of 126 m, and it reaches the rated power output of 5 MW at a wind speed of 12.1 m s −1 .A summary of the most important parameters is listed in Table 1.The wind turbine is a variable-speed variable-pitch machine, such that the rotor speed in belowrated wind is regulated by means of the generator torque, and in above-rated wind by collectively pitching the blades.The natural frequency and damping ratio of the drivetrain dynamics, which are important for the drivetrain damper that will be designed in Sect.4, are ω r = 10.49rad s −1 and ζ = 0.0217, respectively.
The relevant control scheme for the scope of this study is given in Fig. 6.The measured generator speed gen forms the input for the controllers.The torque controller provides a demanded generator torque set point T trq to regulate the rotor speed in below-rated wind, and drivetrain damping can be achieved by superimposing T trq with a small torque ripple T dtd , such that the final demanded generator torque is given by T gen .In above-rated conditions, the generator speed is held constant at ref = rated by means of the collective blade pitch set point θ generated by the CPC.The drivetrain damper and the CPC will be optimized using IFT in Sects.4 and 5.
The software package GH Bladed 4.20 (Garrad Hassan, 2015; Garrad Hassan & Partners Ltd, 2017a, b), which is a certified and widely used wind turbine design software package in the industry, is used to simulate the behavior of the wind turbine in response to a supplied wind field.To do so, the structural model of the turbine is modeled by a multi- body approach combined with a modal representation of the flexible components.The rotor aerodynamics are modeled by combining blade and momentum theory.Initial (baseline) controllers will be constructed using classical loop shaping techniques on linear models of the different subsystems, which are obtained from Bladed using the built in linearization tool.The controllers will be subsequently implemented in the NREL 5MW Bladed wind turbine model and optimized using the IFT algorithm, as Bladed allows for testing of new control algorithms by compiling an external controller designed in MATLAB Simulink (Mathworks, 2013), for example, to a dynamic-like library (DLL) file (Houtzager, 2013).The DLL file is then used during the calculations to obtain the closed-loop dynamics of the wind turbine.The controllers are discretized using the Tustin approximation and are operated at a sampling time of 0.01 s.Note that some of the considered subsystem dynamics are active in a lower frequency range, such that a reduced sampling time would be sufficient.However, in this work all controllers are designed to operate at the sampling frequency, as commercial turbines typically operate at this frequency.The data in the log files are recorded with a sampling time of 0.05 s.The interested reader is referred to the theory manual of Bladed (Garrad Hassan & Partners Ltd, 2017b) for details on the calculation methods.

IFT of a drivetrain damper
Wind turbines that have a geared drivetrain are known to have a lightly damped drivetrain mode.Subjected to turbulent wind, the rotor speed will vary despite the speed regulation by torque control and CPC.The rotor speed variations cause the drivetrain mode to be excited, which can lead to oscillations in the drivetrain.In order to prevent this, a driv-etrain damping controller is typically included in the control system (e.g., see Fig. 6).This controller adds a small torque ripple at the drivetrain frequency (Bossanyi, 2000) to the demanded generator torque of the torque controller.Doing so will dramatically reduce the drivetrain oscillations.Several studies in the past have considered the design of drivetrain damping control, e.g., refer to Bossanyi (2000); Dixit and Suryanarayanan (2005); Fleming et al. (2011Fleming et al. ( , 2013)); Wright et al. (2011).
In this section, the IFT algorithm (see.Sect.2) is applied to the optimization of the drivetrain damper parameters.In the next paragraph, the controller parameterization is given, as well as the classical design approach.Then, a parameter study is carried out to visualize the cost function.Subsequently, the controller is iteratively optimized for realistic loading scenarios as well as for different algorithm settings.Finally, the results of this case study are discussed.

Baseline damping controller design
Typically, the drivetrain controller is chosen to be a bandpass filter of the form (Bossanyi, 2000;Bossanyi and Witcher, 2009) where K is the bandpass gain, ω is the drivetrain frequency, and ζ influences the damping.With a (linearized) model of the drivetrain dynamics G dtd , this bandpass filter can be tuned to damp the drivetrain oscillations.To this end, a linearized model of the drivetrain dynamics is obtained from Bladed using the built-in linearization tool.With this model, the drivetrain damper is designed using classical loop-shaping techniques (Skogestad and Postlethwaite, 2006).The resulting Bode diagrams of the drivetrain dynamics, the open loop (controller times drivetrain dynamics), and the closed loop are shown in Fig. 7. From the openloop dynamics shown in the Bode diagram, it is recognized that the bandpass gain K can theoretically be increased infinitely (i.e., because the phase plot never crosses −180 • ); however, in practice the controller bandwidth is limited by actuator and communication constraints and higher-order unmodeled dynamics.Hence, during the iterative optimization, the controller cannot become unstable (for positive values of the controller parameters).The bandpass filter that is used in Fig. 7 has K = 2500, ζ = 0.3, and ω = ω d = 10.49rad s −1 and is considered as the baseline controller.This filter has been experimentally verified to yield satisfactory performance.

Parameter study for drivetrain damping
Before the IFT algorithm was implemented, first a parameter study using the linearized drivetrain dynamics was carried out.In order to gain understanding of the optimization  problem, the closed-loop system, including the drivetrain dynamics G dtd and the bandpass filter C dtd , is simulated for a range of parameters (K, ω, ζ ).A disturbance signal at the drivetrain frequency is injected at the output.The cost function is taken as Eq. ( 1), i.e., both the output and the input are weighted.The cost function for combinations of K and ω is shown in Fig. 8.The results indicate that the cost function has a large area where it is almost optimal and which becomes wider for increasing ζ .This means that for this problem many combinations of K and ω exist that give almost identical results.Moreover, this also indicates that the parameters are likely to quickly converge to almost minimum cost values, but they will slowly converge to the optimum value.It also shows that the baseline damping filter with ζ = 0.3 could be improved (at least for the simulation case) by decreasing the damping ζ of the filter.Finally, the cost function plot shows that the phase of the controlled system can be adjusted by increasing or decreasing the frequency ω and at the same time increasing or decreasing the bandpass gain K to maintain practically the same compensation performance.

IFT of drivetrain damping
The input of the drivetrain damping controller Eq. ( 31) is taken to be the measured generator speed.The generator speed signal consists of many frequency components arising from the interaction between the wind turbine system and the wind.For drivetrain damping, one is only interested in the frequencies close to the drivetrain frequency.Hence, the controller scheme is taken to be identical to Fig. 4, where C 1 is the torque controller and C 2 is the drivetrain damper.The input to the system is u = T gen and the output is y = gen .
The measured generator speed signal gen is passed through the high-pass filter where ω H = 0.63 rad s −1 is the cut-off frequency and ζ H = 0.7 determines the damping of the filter.In order to make sure that the cost function is dominated by the drivetrain excitations, the output component of the cost function Eq. ( 1) is filtered.The weighting filter with ω r = 10.49rad s −1 at the drivetrain frequency and ζ r = 0.1, is effectively an inverted notch filter passing through the frequencies close to the drivetrain frequency and attenuating other frequencies.The input component of the cost function is also filtered, as one in this case is only interested in the high-frequency part of the generator torque set point T gen .Therefore, the input component is also filtered with the highpass filter Eq. ( 32).The cost function then becomes which thus consists of the measured response of the generator speed gen filtered by Eqs. ( 32) and ( 33) and the demanded generator torque T gen filtered by Eq. ( 32).These filters are also included in the gradient experiment configuration in Fig. 5 just before the controller derivatives.The experiments of IFT for the iterative drivetrain controller optimization are according to Sect.2.4 as follows: First experiment (T : 0-20 s) In the first experiment, the closed-loop system is operated for 20 s (i.e., N = 2000 samples), with C 1 and C 2 identical to Fig. 4. The output gen is filtered by Eqs. ( 32) and ( 33), and the input T gen is filtered by Eq. ( 32).

Second experiment (T : 20-40 s)
The second experiment is identical to the first experiment.
Third experiment (T : 40-60 s) In the third experiment, the filtered output H gen from the first experiment is added to the torque set point T gen and the recorded signals from the second experiment are subtracted as shown in Fig. 5.
After the third experiment at T = 63 s, the controller parameters ρ are updated according to Eq. ( 33), and before the next iteration is started, 7 s is used to let the transients disappear that are caused by the controller parameter update.Thus, a full iteration takes 70 s.In the results, the IFT algorithm is also adjusted to collect N = 1000 and N = 3000 samples (10 and 30 s, respectively) per experiment.In these cases the experiment lengths are adjusted accordingly and the iterations therefore take 40 and 100 s in total, respectively.In all cases, the first iteration starts at T = 30 s after the start of the simulation, such that all initial transients have disappeared.
The generator torque set point T dtd provided by the IFT algorithm is limited to ±1.8 kNm.Moreover, the generator torque set point is also limited in the rate of change, i.e., a maximum rate of ±20 kNm s −1 is allowed.

Results
The results for IFT of the drivetrain damper are subdivided into four parts, each covering a different aspect of the IFT results.

General analysis of results
In the first part, the results for a number of general settings of IFT are analyzed.The wind field considered here has a mean wind speed of 14 m s −1 and a turbulence intensity of 10 %.The number of data points considered is N = 2000, corresponding to 20 s of simulated time (recall that the controllers run at a sampling time of 0.01 s) and the parameter update step size of Eq. ( 2) is set to γ = 0.3.The adjustable signal-to-noise ratio parameter F is set to 2000, which was experimentally found to provide a good trade-off between the signal-to-noise ratio and the amplitude of the injected signal.
First, the convergence of the cost function and controller parameters for different optimization cases are considered.In total four cases are considered: 1.The baseline was considered.In the baseline case the controller parameters are held constant and are given for reference.
3. ρ base,λ was considered.The third case is identical to the second case except that the input weighting is smaller, i.e., λ = 2 × 10 −7 .
4. ρ subopt was considered.In the final case, the initial parameters ρ 0,subopt are chosen to be far from the baseline case: K = 1000, ω = 6.28 rad s −1 , and ζ = 0.4.The input weighting is identical to the second case.
The results of the comparison are shown in Fig. 9. From the three plots related to the cost, it is observed that the baseline controller is already rather close to optimal controller performance.The controller with suboptimal initial controller parameters converges towards the other controller  cases in roughly 10 iterations. 1 It is observed that the optimization of the controller parameters converges in a way that it varies around the steady-state result.The variance of the trajectory is correlated to the turbulence intensity used.The main performance improvement is observed in the generator torque effort.It can be observed that the bandpass gain K and bandpass damping ζ of the baseline controller should, respectively, be increased and decreased in order to improve the performance.The bandpass frequency ω remains close to the drivetrain resonance frequency ω d during the optimization.The influence of the lower input weight-ing λ in the case of ρ base,λ is clearly seen in the bandpass trajectory.
With the controller parameters obtained in the last iteration of the second case, a comparison is made with the baseline case in a normal design load case according to IEC (2005).The frequency spectra of the demanded generator torque T gen and the resulting generator speed gen are shown in Fig. 10.From the spectrum plot of gen it can be observed that the IFT-optimized controller yields a higher damping around the drivetrain frequency ω d .The optimized parameterization also slightly increases the frequency contents around the drivetrain mode.The demanded generator torque T gen displays a similar result.There is slightly more energy concentrated at and around the drivetrain frequency, while at low and high frequencies the energy is reduced.

Impact of F on results
In this paragraph, the influence of the scaling factor F on the IFT performance is investigated.The IFT algorithm is carried out on the drivetrain damper using different values of F .The initial parameters are identical to the baseline and the step size is chosen to be γ = 0.3 and the input weight to be λ = 5 × 10 −7 .The number of data samples collected is again set to N = 2000, corresponding to experiment lengths of 20 s.The turbulent wind field is identical to that in the previous paragraphs.The results for three cases where F is varied between F = 1000 and F = 2000 are shown in Fig. 11.
The first thing that can be observed from the cost function plot is that the cost function slightly reduces with an increase in the scaling factor F .The effect of F on the optimization is more recognized in the convergence trajectories of the bandpass gain K and the bandpass damping coefficient ζ .Clearly, by increasing the scaling factor F , the convergence rate increases.This is also what one could expect from reasoning because the excitation of the system increases with F , as can also be seen from the lower plot in Fig. 11.The choice of F seems to be independent, at least for the considered values, of the final values to which the controller parameters converge.Thus, when applying IFT, the scaling factor should be carefully chosen such that the choice for F in that sense becomes a trade-off between the convergence rate and maximum allowed magnitude of the injected signal.

Varying experiment length N
In the previous results, the length of the experiments in each iteration was N = 2000 (20 s).Here, the iterative optimization results are compared for experiment lengths of 10, 20, and 30 s.The step size and input weight are set to γ = 0.3 and λ = 5 × 10 −7 , and the scaling factor is set to F = 2000.The results are shown in Fig. 12.
It is observed that the experiment length has a clear influence on the variance of the obtained results, both for the cost function plots as well as the controller parameters.The results also indicate that for the drivetrain damping case the experiment lengths do not dramatically change the optimization outcomes.It seems that for the N = 1000 case the load reduction performance is slightly better than in the other cases.Similarly, the N = 3000 case seems to result in the lowest generator torque effort to reduce the drivetrain oscillations.Moreover, for this case the parameters remain rather close to the original baseline parameters.It can be argued that the N = 2000 case provides the best trade-off between the variance, performance, and convergence time.

Varying wind conditions
In the final part of this case study, IFT is also applied to a wind speed of 10 m s −1 at below-rated operating conditions.Although the need for an active drivetrain damper in belowrated varying wind conditions is strongly turbine dependent, this case study will show the ability of the algorithm to optimize controllers while turbine power control is active.For comparison, the 14 m s −1 wind case is also shown.The step size and input weight are once more set to γ = 0.3 and λ = 5 × 10 −7 , and the scaling factor is chosen to be F = 3000 for the 10 m s −1 wind and F = 2000 for the 14 m s −1 wind.The experiment length is kept at N = 2000 samples.The results are displayed in Fig. 13.It is observed in the plots that the convergence trajectories for both wind speeds are very similar.The 14 m s −1 turbulent wind case excites the drivetrain mode more than the 10 m s −1 wind case as is suggested by the increased cost, which thus also requires a higher input energy.The cost function plots show a number of iterations with a clearly higher input cost.This is caused by sudden wind speed changes to which the torque controller responds.Although the input in the cost function is high-pass filtered, these sudden changes remain partly in the input signal.On such occasions, the controller parameters also display relatively large changes.This could be overcome by increasing the cut-off frequency of the high-pass filter that is used to filter the input signal in the cost function or by increasing the experiment length such that these effects are averaged (i.e., the experiment length acts as lowpass filter on the results).

IFT of CPC
This section presents the results of the IFT algorithm applied to step tuning of the CPC.In the next paragraph, first the controller structure is given, including the details of the IFT method thereof.In the subsequent section the optimization results for several cases are discussed.

CPC design and IFT implementation
The CPC of a wind turbine generally consists of a proportional integral (PI) controller cascaded with some filters that prevent undesired contributions in the pitch signal.The full controller of the CPC loop is given by where the filter coefficients are listed in Table 2.The CPC thus includes a notch filter that prevents it from working on the 3P (3 times per revolution) frequency present in the generator speed signal, similarly a notch filter that prevents the controller from reacting to the drivetrain frequency component and a low-pass filter that removes all frequencies above 1.6 Hz are included.The controller takes the measured generator speed gen as input signal and outputs a demanded collective pitch signal θ ; see Fig. 6.The IFT algorithm is applied so as to optimize the step response tracking of the controller in Eq. ( 35).Since CPC is aimed at disturbance rejection rather than reference tracking, this might seem like an odd optimization objective.However, for this single-input single-output case they both target the same sensitivity.Therefore, the frequency-dependent performance is dependent on the excitation signal.Since the power spectrum of a step could be considered as a weighted version of a wind spectrum, with a higher emphasis on the lowfrequency components, meaningful results can be obtained using this approach.This can be further improved by using a reference signal with a power spectrum similar to that of a wind signal.The advantage of the approach followed is that the reference signal is user defined and the wind is uncontrollable.The step change is imposed in the rated genera- Second experiment (T : 0-20 s) In the second experiment, the closed-loop system is operated with ref = rated .The output gen and process input θ are recorded and used in the next experiment.
Third experiment (T : 65-85 s) In the third experiment, the gradients are obtained by operating the closed-loop system with ref = rated , injecting the error signal F ( ref − gen ) from the second experiment at the process input θ , using the recorded signals of the first experiment at the process input of the system according to Sect.2.3, and filtering with controller derivatives (including the high-pass filter for the input gradient).
The time between the second and third experiments is required to make sure all oscillations due to the step change have disappeared.At T = 86 s, the controller parameters are updated.Then, after 34 s during which the transients due to the gradient experiment and the controller update have disappeared, at T = 120 s, the next iteration is started.The first iteration starts at T = 30 s.During the optimization, the maximum pitch rate θ is limited to ±8 • s −1 .Notice that the order of the first and second experiments have been reversed during the optimization in comparison to Sect.2.3.The cost function is chosen as where the input weighting factor λ and the step size γ in the parameter update rule Eq. ( 2) are both considered for different values.Note that the input signal in the cost function is high-pass filtered by L θ , which is identical to Eq. ( 32).The high-pass filter L θ is required because the optimization procedure should focus on the dynamic pitch response rather than on the static pitch offset required to maintain the rated generator speed.

Results of IFT for CPC
The first result is obtained by optimizing the CPC for a turbulent wind field with a mean wind speed of 14 m s −1 and turbulence intensity of 4 %.The step change for this result is chosen to be step = 30 rpm.Moreover, the adjustable signal-to-noise ratio gain F is set to 0.02.The initial PI controller values are K p = 4 × 10 −3 and K i = 1 × 10 −3 .The results for three cases with varying input weighting λ and step size γ are shown in Fig. 14.The trajectory of the cost function values and the controller parameters are shown in Fig. 14.Note that the performance objective in this work is expressed by the cost function J (ρ), and minimization of this function is considered as improved performance.Another method for controller performance comparison is the use of Pareto curves (Odgaard et al., 2015a, b), but this technique will not be considered.
It can be observed that the initial controller parameters were suboptimal and they converge in a few iterations to a much better performance.The results also show that the con-vergence of the parameters behave differently.The proportional gain K p converges to its final value after four iterations.Conversely, the integral controller gain K i slowly increases to larger values.The difference in trajectory can be explained due to the fact that in the step response the proportional gain is more dominant.In order to make the integral controller parameter more dominant, one could increase the experiment length N.
The effect of the gains γ and λ on the convergence trajectories are apparent in Fig. 14.It is observed that the convergence of the proportional gain K p for the cases where λ = 5 × 10 3 is faster due to the higher step size γ .The effect of increasing the weight λ in the cost function is also clear from Fig. 14: the parameters converge to smaller values, which is expected.
In Fig. 15, three different step responses are shown.The blue graph displays the response of the initial controller to the step change.As can be observed, this response is sloppy and after nine iterations has improved to a decent response.The increased weight on the input cost yields a step response where less pitch duty is required with only a very limited loss of tracking performance.In Fig. 16, the generator speed response and the collective pitch angle for a full iteration during the first and ninth iterations are displayed.
The final results involve a comparison between IFT of CPC for two different wind speeds: 14 and 18 m s ther pitched from the wind, which means that less pitch effort is required to keep the rotor speed close to rated.The IFT tuning results also display this behavior.In Fig. 17 it can be seen that the proportional gain for 18 m s −1 is roughly two-thirds of the value compared to the 14 m s −1 case.Conversely, the integrator gain K i is somewhat higher for the 18 m s −1 wind speed.The cost function converges to a comparable result.The step responses shown for the seventh iteration are also rather similar.

Conclusions
In this paper, IFT controllers for wind turbines have been developed.The typical controller configurations used for wind turbine control require three closed-loop experiments to be carried out.With the data that are collected during these experiments, it has been shown that IFT can be successfully ap-plied.The results indicate that starting the optimization from a baseline controller with decent performance can already improve the performance within a few iterations.It has also been shown that IFT can be applied to both disturbance rejection and reference tracking control for wind turbines.This is demonstrated by means of optimizing the drivetrain damping controller and the CPC.The methodology could similarly be applied to improve fore-aft and/or side-side tower damping performance.Finally, it is argued that IFT could be a valuable tool with which the performance of wind turbine controllers can be improved without the need of system identification.
Data availability.Simulation data and code used in this paper are available under doi:10.5281/zenodo.345978,(van Solingen et al., 2017).

Figure 1 .
Figure 1.Block diagram of a closed-loop system.

Figure 2 .
Figure 2. Block diagram of the closed-loop gradient experiment.
Solingen et al.: Iterative feedback tuning of wind turbine controllers

Figure 3 .
Figure 3. Block diagram of modified closed-loop gradient experiment for reference inputs with offsets. ∂u

Figure 4 .
Figure 4. Block diagram of the closed-loop system, which involves multiple controllers (representing the scenario of torque control and drivetrain damping).

Figure 5 .
Figure 5. Block diagram of the gradient experiment for the multiple controller scenario.The controller C 1 is fixed and controller C 2 (ρ) is subject to IFT.The signals Hy 1 (k, ρ), y 2 (k, ρ), and u 2 (k, ρ) are obtained from two other experiments and are required to obtain unbiased gradient estimates.

Figure 6 .
Figure 6.Overview of the relevant controller configuration in this study.The starred controllers will be the subject for IFT.

Figure 7 .
Figure 7. Bode diagrams of the drivetrain dynamics, the open-loop (controller times drivetrain dynamics), and the closed-loop system.

Figure 8 .
Figure 8. Close-up of the optimal parameter combinations for the drivetrain excitation problem with ζ = 0.1 (left) and ζ = 0.3 (right).The results are thresholded and normalized for clarity.The results show large areas where the cost function is almost optimal.The minimum values of the cost function for each case are indicated by the white + marker.

Figure 9 .
Figure 9.Comparison of three drivetrain torsional damping controllers, two of which are subject to IFT.The baseline controller parameters are kept constant and the result is shown as reference.The other three cases involve IFT, where the torsional damper is optimized starting from different initial conditions.Results shown are obtained with γ = 0.3, F = 2000, and turbulent wind with mean speed 14 m s −1 and 10 % turbulence intensity.

Figure 10 .
Figure 10.Comparison of the baseline and IFT-optimized drivetrain controller performance.

Figure 12 .
Figure 12.Comparison of IFT performance for different number N of collected data samples.Results shown are obtained with γ = 0.3, λ = 5 × 10 −7 , F = 2000, and turbulent wind with mean speed 14 m s −1 and 10 % turbulence intensity.

γFigure 14 .Figure 15 .
Figure 14.Reference step tuning using IFT of CPC at a wind speed of 14 m s −1 with 4 % turbulence intensity.The results show three different cases where the step size γ and the input weighting λ in the cost function are varied.

Figure 16 .
Figure 16.Comparison of responses during the first and ninth iterations.Results shown are obtained for a wind speed of 14 m s −1 with 4 % turbulence intensity.The measured generator speed and reference input are shown in the upper plot, and the lower plot displays the pitch response.

Figure 17 .Figure 18 .
Figure 17.Reference step tuning of CPC using IFT of CPC at a wind speed of 14 m s −1 with 4 % turbulence intensity.

Table 2 .
Filter parameters of the different controller components.