Generating expression templates for algorithmic differentiation on domain specific languages

The benefits of domain-specific languages in software development are usually very pronounced. Domain-specific problems can be formulated faster, and domain experts need less training to formulate the problems in software. In addition, the software runs faster since the underlying library can implement specialized algorithms and can be adjusted to the hardware. Applying a general operator overloading algorithmic differentiation tool in such a setting can be quite tricky. The replacement of the underlying computational floating-point type with the specialized type of an AD tool has two problems. First, the memory structure of the program is changed, and floating-point data is interleaved with identifiers from AD. This prevents the compiler from performing optimizations such as SIMD optimizations. Second, the AD tool does not see any domain-specific operations, e.g., linear algebra operations, that the program uses. This prevents the AD tool from using specialized algorithms in such places. Applying a specialized operator overloading AD tool can eliminate these drawbacks, but the AD tool needs to understand the DSL. This would mean that the AD tool knows the objects of the DSL, e.g., vectors and matrices, and the operations, e.g., matrix-vector product. The major challenge is then the annotation of the DSL for the AD tool. For all operations, the derivative must be found and written down so that the AD tool can use it. For a maintainable solution, these definitions should be independent of the AD tool internals and close to the definition of the primal computation. In addition, the extra code should not be too large, and it should not influence runtime and compile time when AD is not used. The target language for the AD tool is C++, which means this information needs to be provided at compile time. Runtime reflection is therefore not an option. We want to explore how the annotations can be designed such that the above criteria are meet. In addition, the generation process for the AD-specific code parts is explored in more detail.