Re: [eigen] Efficiently creating a matrix of pairwise vector differences. |

[ Thread Index | Date Index | More lists.tuxfamily.org/eigen Archives ]

*To*: eigen@xxxxxxxxxxxxxxxxxxx*Subject*: Re: [eigen] Efficiently creating a matrix of pairwise vector differences.*From*: Jeff Hammond <jeff.science@xxxxxxxxx>*Date*: Sun, 24 Dec 2017 16:58:15 +0000*Dkim-signature*: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=kXcB5rTdeeI24cNVlLZ48U5iu2CBWswItoOudqAmhnc=; b=C9sOfKwcBF4Txy5xFdPFVfsnijFnkQZGCzrIdKfSTYV3AJLmnZOwvFrWrsUC77MU4E YDSrzTtg7I/J6dJGp1C2NiJqzXoelDHoQSGe1ZxeQPyc7Tsbcypx7mZna0zlud2Z1ek5 xVTFOGV+cGENxXdUkLQNt6gtmX1K9GJVzkOdrOjYmx1UT85YdvLNZMGHRQtWVi7smNbd k8AZ/JkLXe0NE98FK+jZPDMfpxM69a+Q8o989XQRKfjmaz66xvUCbNga100LYGljPQEe 2gUHzoClNlZ0rJQUFTYj7NaQHA+1RA4DRpm0TvdZeLyBO00w9RmMD2T2oGC6M+53DmzN jsOw==

On Wed, Dec 20, 2017 at 3:44 PM Smith, Louis <Louis_Smith@xxxxxxxxxxxxxxxxxx> wrote:

Hello,

I'm trying to use eigen to compute the distances between m length vectors i and j which are each rows in an NxM matrix (note that M is often much much smaller than N. In my test case N is about 250,000 and M is 6). What I'm currently working with is an _expression_ like:

Matrix Xd data = "" //This works when data is written to cout, so elided.

MatrixXd distances = (data.rowwise() - data.transpose().colwise().transpose()).norm();

Which gives me the following error:

error: no member named 'transpose' in 'Eigen::VectorwiseOp<Eigen::Transpose<Eigen::Matrix<double, -1, -1, 0, -1, -1> >, 0>'

MatrixXd distances = (data.rowwise() - data.transpose().colwise().transpose()).norm();

When I get rid of the third transpose I'm then subtracting a column vector from a row vector, which also doesn't work. Calling transpose on the row vector gives a similar error.

What I expect distances to be is an NxN symmetric matrix containing the distances (norms of difference vectors) for each pair of row vectors in data.

Sorry for the newbie question, but I'd really appreciate some insight on this since it seems like there should be a more eigeny way to write this than the double-for loop over the data, which also works but is very slow.

If you’re writing a 250000x250000 matrix, I don’t know how fast you think it should be. If you did a GEMM between 250000x6 and 6x250000, it is likely limited by write bandwidth, because the inner loop is 1-6 cycles on modern hardware. You can compare GEMM to triple looks for these dimension to verify.

I suspect double loops is actually going to win, especially if you convince the compiler to generate nontemporal stores (assuming x86) to minimize RFO traffic..

Jeff

--

**References**:**[eigen] Efficiently creating a matrix of pairwise vector differences.***From:*Smith, Louis

**Messages sorted by:**[ date | thread ]- Prev by Date:
**Re: [eigen] Efficiently creating a matrix of pairwise vector differences.** - Next by Date:
**[eigen] OCaml numerical library and bindings** - Previous by thread:
**Re: [eigen] Efficiently creating a matrix of pairwise vector differences.** - Next by thread:
**[eigen] SparseMatrix StorageIndex issue**

Mail converted by MHonArc 2.6.19+ | http://listengine.tuxfamily.org/ |