>
> gael.
>
>>
>> Benoit
>>
>> >
>> > gael.
>> >
>> >
>> > On Thu, Mar 4, 2010 at 10:43 PM, Benoit Jacob <
jacob.benoit.1@xxxxxxxxx>
>> > wrote:
>> >>
>> >> 2010/3/4 Gael Guennebaud <
gael.guennebaud@xxxxxxxxx>:
>> >> >> >> > alloc:
>> >> >> >> > m_data = ei_aligned_new(size+<16 bytes>) + <16 bytes>;
>> >> >> >> > allocatedSize() = size;
>> >> >> >> >
>> >> >> >> > dealloc:
>> >> >> >> > ei_aligned_delete(m_data-<16bytes>);
>> >> >> >> >
>> >> >> >> > int& allocatedSize() {return (m_data-<16bytes>);}
>> >> >> >> >
>> >> >> >> > Disclaimer: yes the above is not C++, it is just to picture the
>> >> >> >> > idea!
>> >> >> >>
>> >> >> >> This looks like going only halfway toward heap-stored data.
>> >> >> >> Instead,
>> >> >> >> why not take the bolder move of adding a d-pointer? We would put
>> >> >> >> there
>> >> >> >> any additional data that is OK to access with non-inline
>> >> >> >> functions.
>> >> >> >> So
>> >> >> >> we would keep directly as data members the array pointer m_data
>> >> >> >> and
>> >> >> >> the dimensions m_rows and m_cols so we can still call
>> >> >> >> rows()/cols()/data() at zero cost (useful as they are used all
>> >> >> >> the
>> >> >> >> time) but other less frequently used data could be deferred onto
>> >> >> >> the
>> >> >> >> d-pointer and accessed through no-inline accessors.
>> >> >> >
>> >> >> > To be honest I don't see how adding a d-pointer can offer more
>> >> >> > flexibility.
>> >> >>
>> >> >> With your approach, any additional member data that we may want to
>> >> >> add
>> >> >> in the future, has to fit in the fixed number of bytes that were
>> >> >> reserved, like 8 bytes or 16 bytes. We have to decide once and for
>> >> >> all
>> >> >> how much space we reserve for additional members.. Moreover, once
>> >> >> we've
>> >> >> added a member, we have to keep its offset fixed forever. All of
>> >> >> that
>> >> >> can theoretically be overcome by using a d-pointer.
>> >> >
>> >> > hm... maybe I've been clear but as "my proposal" I was referring to
>> >> > the
>> >> > solution of storing the D_structure in the dynamically allocated
>> >> > memory,
>> >> > i.e., with the data.
>> >>
>> >> Well let's look at your pseudo code:
>> >>
>> >> m_data = ei_aligned_new(size+<16 bytes>) + <16 bytes>;
>> >> allocatedSize() = size;
>> >>
>> >> dealloc:
>> >> ei_aligned_delete(m_data-<16bytes>);
>> >>
>> >> If I understand correctly, you're reserving a fixed amount of memory
>> >> (here 16 bytes) for the D_structure just before the location pointed
>> >> to by m_data. So yes it's on the heap, that's what I understood, but
>> >> you still hardcode the number of bytes that your D_structure may have.
>> >>
>> >> >> > My proposal affords the same with less memory and runtime
>> >> >> > overhead: a
>> >> >> > true
>> >> >> > d-pointer would requires in addition one pointer, one call to
>> >> >> > malloc,
>> >> >>
>> >> >> Yep, I thought about that just after sending the e-mail. The
>> >> >> solution
>> >> >> might be to merge this idea with your idea: allocate at once the
>> >> >> matrix array and the D_structure. But in order to allow the
>> >> >> D_structure to grow in the future, place it after the array, not
>> >> >> before, and access it only with non-inline accessors .... now here's
>> >> >> the catch... that must be compiled into a shared library :( I didn't
>> >> >> think about that in my previous e-mail, but the d-pointer approach
>> >> >> can
>> >> >> only work if we have a binary shared library :( Though at that
>> >> >> point,
>> >> >> having such a tiny library would solve a bunch of problems at once
>> >> >> (cache size parameters, etc). I don't know what to think about that.
>> >> >
>> >> > You cannot easily put it at the end because ideally we would store
>> >> > the
>> >> > allocatedSize variable in the D_structure, and if you put it at the
>> >> > end
>> >> > of
>> >> > the data, you need the allocatedSize to access to the D_structure...
>> >>
>> >> Well yes, in my proposal of putting the D_structure at the end, we
>> >> have to add a new data member to Matrix, which can be either the
>> >> offset or why not directly the pointer to the D_structure.. But I don't
>> >> think that it should be the allocatedSize that we should store, and
>> >> actually it still wouldn't be too convenient to address the
>> >> D_structure (need to take padding into account...)
>> >>
>> >> Then, from the moment we're storing 2 pointers, m_data and m_d, it
>> >> doesn't matter anymore which one is at the beginning and which one is
>> >> at the end of the buffer.
>> >>
>> >> Is that a big deal to add one more data member to MatrixXf...?
>> >>
>> >> Though in that vein, one might go further and ask why we're preferring
>> >> to put stuff on the heap at all, why not just add plain data members
>> >> to MatrixXf...? I'm not sure why sizeof(MatrixXf) matters more than
>> >> the size of the allocated buffer.
>> >>
>> >> > Since
>> >> > this whole approach can only work via a shared library,
>> >>
>> >> ...if we want a real d-pointer. Without a shared lib, we can still
>> >> have a D_structure, it's just that the application using Eigen
>> >> hardcodes the D_structure data layout at compile time, so we don't get
>> >> the flexibility of a d-pointer.
>> >>
>> >> >> I'm completely hesitating, I can't make a decision on that. I guess
>> >> >> that if we treat this issue simultaneously with other issues that
>> >> >> would benefit from a binary lib, such as cache size runtime
>> >> >> parameters, then the case for a binary lib gets quite strong. On the
>> >> >> other hand it will require good communication and documentation, it
>> >> >> would be great to keep it optional (maybe make its code optionally
>> >> >> available as a header file...), and it should be WTFPL-licensed.
>> >> >
>> >> > Same here, though I become more and more in favor to a shared library
>> >> > as
>> >> > it
>> >> > might solve many issues.
>> >>
>> >> i don't know... above we're discussing a very good solution without a
>> >> binary lib, and below you have a great idea for the cache size problem
>> >> too:
>> >>
>> >> >
>> >> > Regarding runtime settings without a shared lib, I was thinking about
>> >> > using
>> >> > a static variable inside a function:
>> >> >
>> >> > // internal
>> >> > int manage_cache_size(enum action,int v=0)
>> >> > {
>> >> > static int value = EIGEN_DEFAULT_CACHE_SIZE;
>> >> > if(action="" value = v;
>> >> > if(action="" return value;
>> >> > return value;
>> >> >
>> >> > }
>> >> >
>> >> > // public:
>> >> > int cacheSize() { return manage_cache_size(get); }
>> >> > void setCacheSize(int v) { manage_cache_size(set,v); }
>> >> >
>> >> > but I'm really unsure about that...
>> >>
>> >> wow, that looks like a great idea!
>> >>
>> >> Such a static variable in a function, works exactly like a global
>> >> variable from a library as far as we're concerned... as far as I can
>> >> see.
>> >>
>> >> Benoit
>> >>
>> >>
>> >
>> >
>>
>>
>
>