Most businesses start exploring AD with a Proof of Concept (PoC) assigned to one, or sometimes two, ‘in-house’ developers. Very often, however, these PoCs don't fare very well because:

• the developer is new to (A)AD and has to learn a lot quickly, and usually on his/her own
• the developer must learn how to control memory use efficiently for production-size codes (our tools make this easy, but doing it effectively requires a good  understanding of AD concepts and of the underlying code)
• typically, the PoCs are very time-limited: the developer must report back to the business by a given date, schedules are tight, and any hiccups or problems mean the schedules quickly slip
• the developer needs to learn how to use our AD tools: many people don't have the time to read through our documentation and example programs
• the developer often gets pulled onto other more urgent business matters mid-way through the PoC, then has to come back later on and try to  remember where he/she was
• once the AD code is working, the developer then needs to explain results: e.g., why are there zero derivatives with respect to some parameters when finite difference estimates are non-zero?  Typically this is because the underlying code is not differentiable: ideally, this should be spotted right at the outset and addressed before AD is applied

This is quite a daunting list. NAG's AD Proof of Concept Support service has grown through client demand: in a nutshell, NAG experts conduct the PoC with the client.  

• The organisation takes one of our AD developers in-house, typically for a week.
• Our developers work with the organisation's to get the entire PoC up and running as quickly as possible
• It's a combination of coding help and on-the-job training: we answer questions, point out pitfalls, explain AD concepts, inspect the code for non-differentiabilities, point out where analytic adjoints are desirable, and generally offer advice on design, implementation and testing. 
• The PoCs are completed much faster, developers learn faster because there's someone next to them to answer questions, and the resulting AD code is efficient and has predictable memory use. 

NAG Proof of Concept AD Support Results

For an organisation new to AD, the NAG Proof of Concept AD Support service allows a rapid deep-dive into what AD means for your business, with the security of knowing there are experts at hand who make sure you get the answers you need, on time.  By the end of the PoC, the developers can not only report results back to the business with confidence, but they will be familiar with our tools and will have a non-trivial reference code from which to start for the next project.

NAG provides best-in-class C++ operator-overloading AD tools called dco (derivative computation through overloading) and dco/map (dco meta adjoint programming).

• dco/c++ integrates easily with your code base, is extremely flexible, and has been applied to huge (millions of lines) production codes
• dco/c++ allows the memory use of adjoint codes to be constrained almost arbitrarily
• dco/map is a cutting edge C++11 tape free operator overloading AD tool designed specifically to handle accelerators (GPUs, Xeon Phi, etc).  An overview is available here: New Technical Poster: High Performance Tape-Free Adjoint AD for C++11
• dco/map combines the benefits of operator overloading (easy application, one single source code) with the benefits of handwritten adjoints (very fast and minimal memory use)
• dco/map can handle the race conditions inherent in parallel adjoint codes easily and efficiently
• dco/c++ and dco/map are easily coupled to create AAD implementations of heterogeneous codes

NAG AD Software Tools are very efficient. However, it is sometimes desirable to produce an AD implementation in a different way, often to handle computationally expensive sections of code. Typically, this is through hand-writing an adjoint implementation, or using a high-performance tool such as dco/map to make the AD implementation, or even porting the AD code to a GPU. NAG AD Solution services can assist in all these cases, whether it be writing adjoints for particular pieces of code, or porting to GPU and making GPU adjoint implementations. 

Case study highlights: The normal simulation on a top-end desktop took 44s, while the AD-enabled simulation took 273s. To obtain the same gradient information, on the same machine, by finite differences would take roughly 5 years.

Figure 1: Sensitivity of drag coefficient to the surface geometry of a car at high speed (left) and low speed (right)

AD enables sensitivity analyses of huge simulations, enabling shape optimization, intelligent design and and comprehensive risk studies.

Figure 1 shows sensitivities of the drag coefficient to each point on a car's surface when it moves at high speed (left) and low speed (right). The simulation was performed with AD-enabled OpenFOAM built on top of the AD Software Tool dco. The surface mesh had 5.5 million cells and the gradient vector was 18GB.

The normal simulation on a top-end desktop took 44s, while the AD-enabled simulation took 273s. To obtain the same gradient information, on the same machine, by finite differences would take roughly 5 years.

The gradient information can now be used to optimize the shape of the car so as to reduce the drag

Towara M and Naumann U (2013). A discrete adjoint model for OpenFOAM. Procedia Comp. Sci. Volume 18.