Re: [eigen] Malloc-free dynamic matrices

[leon zadorin <leonleon77@xxxxxxxxx>]

On 3/5/10, leon zadorin <leonleon77@xxxxxxxxx> wrote:
> On 3/5/10, Gael Guennebaud <gael.guennebaud@xxxxxxxxx> wrote:
>> yes there are hundreds of way to achieve these goals without touching
>> Matrix, but none of the solutions will be as convenient as having it by
>> default directly inside Matrix. For instance, nobody would ever think
>> about
>> removing reserve from std::vector! I really don't see any valuable
>> argument
>> against doing so.
>
> I am not quite sure if we are talking about the 100% same thing here.
>
> The original poster's very goal (and his code examples) for
> 'NeverRealloc' appear to be semantically different from the concept of
> "reserve".
>
> In fact:
>
> ::std::vector<float> v1, v2;
> v1.reserve(10), v2.reserve(100);
>
> // populate v1 and v2 with 10 and 100 elements respectively
>
> v1 = v2; // the original poster wants an assertion/error to be
> generated -- i.e. by *design* he wants to *never* reallocate -- hence
> the "NeverRealloc" option name... -- as per his own code example...
>
> I know that you, later in your email, re-qualify this as an optional
> feature -- but if so, to me, this is rather different w.r.t. the
> concept of just reserving/caching the memory.
>
> For instance, if we have already started comparing/justifying concepts
> of "reserve" in eigen's matrix based on reservation semantics in
> ::std::vector -- should then one also, conversely, consider having the
> 'assert/error' option (when re-growth may be needed at the target obj)
> for the ::std::vector as well? Of course not... :-)
>
> "Reservation != reallocation"

by bad -- a typo, somewhat:

"Reservation != no-reallocation" ...

> and clumping both into the same context
> is not real clear to me.
>
> So, if we are talking about "reservation"  -- then *yes*, of course,
> such a feature would be convenient in the Matrix obj. -- and it can
> lead one on the path for more elaborate caching later on (e.g. not
> just reserving-ahead via "reserve" but also having heuristics for
> keeping the memory-size from previously-used large resize state
> (similar to growth-control for large strings in Mozilla browser
> afaik), etc. etc. etc. -- be those ideas reasonable/needed or not :-)
>
> ... but this would be a rather different context for the conversation
> as compared to the original's idea of *never* realloc and generating
> assertions for the matrix re-growth...
>
> [...]
>
>> And finally, remains the question whether resizing above the pre
>> allocated
>> size should be allowed or not. Here it is pretty clear that the default
>> must
>> allows that.
> [...]
>> What do you think?
>
> I agree indeed -- if one does go ahead with the "reserve", then
> asserting is not a good idea (at least with a default state... if
> ever) -- as "reserve" and "never-realloc" are two *distinct*
> *different* semantic points -- otherwise ::std::vector would have
> these assertions enforced as well :-)
>
> leon.
>
>> On Thu, Mar 4, 2010 at 3:58 AM, leon zadorin <leonleon77@xxxxxxxxx>
>> wrote:
>>
>>> On 3/4/10, Márton Danóczy <marton78@xxxxxxxxx> wrote:
>>> > On 3 March 2010 15:44, leon zadorin <leonleon77@xxxxxxxxx> wrote:
>>> >> double * reserved(aligned_alloc(8192));
>>> >>
>>> >> // 3x3
>>> >> Map<double> m(reserved, 3, 3);
>>> >>
>>> >> // 10x10 -- resize w/o reallocation -- equivalent to NeverRealloc w/o
>>> >> much repatching(?)
>>> >> m.remap(reserved, 10, 10);
>>> >
>>> [...]
>>> > Matrix<float, Dynamic, Dynamic, NoRealloc> A, B, C;
>>> >
>>> > A.reserve(16); A.resize(4,4);
>>> > B.reserve(25); B.resize(5,5);
>>> > C.reserve(81); C.resize(9,9);
>>> >
>>> > B=A; //this should work and resize B to 4x4
>>> > B=C; //this should raise an assertion failure
>>> >
>>> > To achieve this behaviour, we do need a dedicated class -- or an
>>> > option, such as in the patch.
>>>
>>> Sure... but not quite -- if we are talking about adding extra
>>> "sanity-checking" in the debug builds of an app, and if current API
>>> has no such assertions, then some form of changes would be needed.
>>>
>>> But in such a case, still -- I don't think Benoit was arguing against
>>> any changes -- only in favor of smaller/more-elegant ones...
>>>
>>> Map<> obj. ctor could indeed have an additional arg denoting the max
>>> size of the (re)mappable mem (on which it is mapping)... or indeed, as
>>> others have alluded to, one could pass an 'array object' instead of
>>> just 'float/double *' (given that what you are really trying to
>>> 'sanity-check' is the boundary -- and such is not meant to
>>> auto-reallocate but to assert -- implying that this should not be
>>> happening 'by design') -- so a logical conclusion would be for Map<>
>>> to be able to map from a vector/array type (e.g. ::boost::array
>>> semantics), then:
>>>
>>> array a1(25);
>>> array a2(81);
>>>
>>> Map m1(a1, 5, 5);
>>>
>>> Map m2(a2, 9, 9);
>>>
>>> m1 = m2; // assertion due to boundary-checking of 'intelligent'
>>> arrays...
>>>
>>> or if constructing/mapping explicitly from "double/float *"
>>>
>>> double *a1(alloc(25));
>>> Map m1(a1, 5, 5, 25); // i.e. additional arg denoting max mappable mem.
>>>
>>> double *a2(alloc(81));
>>> Map m2(a2, 9, 9, 81);
>>>
>>> m1 = m2; // the same assertion... (ifndef NDEBUG of course)
>>>
>>> In fact, given that you are only requiring an assertion (not exception
>>> throwing), then the actual presence of the above fields (e.g. in the
>>> Map class, the 'bounds-keeping' variable could, itself, exist only
>>> #ifndef NDEBUG
>>> size_t max_mappable_size;
>>> #endif
>>>
>>> Alternatively, as you mentioned -- another class may be used... but,
>>> if such is to be the case, then what I was trying to say was that this
>>> class feels like being more of a facade design-pattern, not a patch to
>>> the existing class:
>>>
>>> struct blah {
>>> double * data;
>>> Map m;
>>> blah()
>>> : data(allocate), m(data, x, y)
>>> {
>>> }
>>> etc. etc. -- like op= with checking for memory-bounds violations...
>>> };
>>>
>>> ... but only if one wants it...
>>>
>>> from your code-example it looks to me like it really should be subject
>>> to an "array" object... or a Map object... as memory-bounds checking
>>> is really a question of abstracting the c-style 'arrays' in the first
>>> place... ideally...
>>>
>>> kind regards
>>> Leon.
>>>
>>>
>>>
>>
>