|Re: [eigen] cppduals - dual number implementation with Eigen specializations|
[ Thread Index |
| More lists.tuxfamily.org/eigen Archives
- To: eigen@xxxxxxxxxxxxxxxxxxx
- Subject: Re: [eigen] cppduals - dual number implementation with Eigen specializations
- From: Michael Tesch <tesch1@xxxxxxxxx>
- Date: Tue, 3 Dec 2019 17:11:21 +0100
- Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=EF+fR3XJGcUSnrPdOVLLHFYXM5zsLHcJ+sFlnspGalI=; b=vPo5b68w90yIrGqIdimHOj4wgEoCfmXxk6ILPeScLHcDkq6nFJcFeb/rnG9JsIrKHn pfSPN9DBxBwDNCSP37udBzriz/3XfwQG3/XIkt9Ym3HAn2LlOWRXqDUlybZ7ds1MaK89 HTBIvpA0i3zp2P12nWcHQhmQSn0vEVuqcsenxsfNjf6ePRwAbNvXZ3WszFByCCuKs3wm T1otiyd338bp5MutLOxSx6GEGkO9CCYhQPOKYkJeRCr7OktO5ee8ChhPBIjwpUJK1Esn 7PkkGz6wrOXD5BKNQgK8eRcm6i7HNgX34MmptkK1OtNrSAKJxi5sfGthYZ9o6BdLCLaQ dF8Q==
Yes, it's only first derivatives.
It would be possible to add second (and third) derivatives, but because the main point is taking advantage of SSE/AVX registers and ops, it sort of awkwardly makes most sense for things that fall into multiples of two (val, derivative) (val, first, second, third).
It would be great to add higher orders and multivariate support, but I dont anticipate my Copious Free Time enabling that anytime soon. If someone wanted to chip in, that would be fancy. There are several similar libraries that have those; the unique thing here is the explicit vectorization and leveraging Eigen's cache-optimized matrix ops.
That's great, I'll check out your implementation. This still only gives first derivatives, right? Any plans for multivariate derivatives?
Yes, of course. It's mentioned in the paper. The dual approach is faster (one multiplication fewer per op) and exact (doesn't rely on +h). The only place where the complex-step might be better is when a) your function is real, and b) you dont need much accuracy, and c) you have BLAS's complex-complex matrix operations that are faster than Eigen's optimization of the dual-valued matrix ops.
There is/was some performance problem with Eigen's optimization of mat-mult of complex-valued matrices (and by extension, my dual-valued Eiegn matrices). The real-valued operations are/were very close to OpenBLAS on my machines, but the complex valued ops are quite a bit slower. I mentioned it on the Eigen chat room once, but I didn't have time to track it down and come up with a proper bug-report. (sorry, it will happen some day.. in case it hasn't already been fixed.)
I've written (yet another!) Dual Number implementation for automatic differentiation. It is meant to be used as the value-type in Eigen matrices, and has templates for vectorization (shockingly) similar to (and based on) Eigen's complex-type vectorizations. It is quite fast for first-order forward diff, and imho pretty easy to use. There are also SSE/SSE3/AVX vectorizations for std::complex<dual< float | double >> types.
I hope this could be useful for someone and would be glad for any feedback, improvements, etc.
It would be interesting to compare this approach to others, by hand-wavey arguments I believe it should ultimately be faster in certain cases.