Sunday, June 17, 2012

Symbolic differentiation with subexpression sharing

This week Yingfu and Adam started working on an implementation of symbolic differentiation for expressions in Scala. A differential operator implementation is ready for a small expression type including a few operators. An interesting part of the implementation is the subexpression sharing mechanism. Here, an "expression context" is used to maintain a collection of references to intermediate expressions (corresponding to a list of let bindings). Constructors of the expression types are not called directly in the program. Instead, factory methods are used for instantiation, which first look in the expression context to see if an existing instance can be referenced. A simple way to implement the expression context is to use a Scala Map object. Here, it is possible to leverage the default implementations of the hashCode method which come with Scala case classes, making it possible to use the expressions themselves as keys in the map.

As a concrete example, consider differentiating the expression x^2 + x^2 with respect to x. The result of applying our symbolic differentiation operator is the following (in pseudocode):

let
  x_4 = Plus(x_0,x_0)
  x_1 = Times(x_0,x_0)
  x_5 = Plus(x_4,x_4)
  x_2 = Plus(x_1,x_1)
  x_3 = Literal(1.0)
  x_0 = Variable("x")
in
  x_5