Re: ISO size - Tazpkg files.list and md5

[GoKhlaYeh <gokhlayeh@xxxxxxxxxx>]

On Sun, 23 Jan 2011 19:12:09 +0100
Christophe Lincoln <pankso@xxxxxxxxxx> wrote:

> 
> Hi devs,
Hi

> Looking into the core ISO I found we have a very HUGE 3,7 Mb
> for /var/lib/tazpkg/installed. It is to BIG and we have duplicated
> informations. We have for each packages a files.list and a md5sum list
> of files... so we could get ride of the files.list and save 1 Mb for
> the core ISO. Someone motivated to modify tazpkg and tazwok before 4.0 ?
files.list and files.md5 doesn't contains exactly the same info, as
files.list also contains folder and symling, which are not handled by
md5. Having symlink datas is usefull when updating/removing a package,
and to look for update into libs (auto-recook-depends). So, light this
is possible by removing from files.list all files already in files.md5;
but removing files.list entirely is not possible. Doing that means at
least:
* Change the packing function into tazwok
* Change the gen_list to, as it usually use files.list to get the files
* Change install, remove, extract, list-files, all which check for file
override & save modifiers(like volatile.taz.*); Make no errors appears
if there's no files.list

That's entirely possible an desirable, but a little heavy for me at
this moment.

> Finding also the gui a bit slow when a lot of packages are installed I
> thinks we could use an SQLite DB for all packages info. Finding info in
> a SQL db is faster than flat files no ?

SQL data management is faster than using commons linux tools & plain
files. There's some point to take care :
* We can't use SQL-only management, as not every SliTaz system has
SQLite installed - so it means we have to duplicate functions -
one for handling things using SQLite, another to handle plain text
(i.e.: plain-text db generated by SQLite) and a little piece of code to
check which one to use (check for sqlite bin). This can be a really
good improvement in SliTaz tools - I thought about that because
generating some files used by tazwok-experimental to order cooklist
take few minutes. 
* About the tazpkgbox specifical case: I havn't read the code but the
packages.desc syntax is a gtkdialog grid syntax, so I guess that this
file is used directely as it is to load the packages grid. Apparently
it also checks for installed packages to load the right icon; So, I
guess that sqlite usage will not be so usefull as you need a text-file
using the right syntax anyway to load the grid (can someone correct me
if I'm wrong ?)
* There are some other way to speed-up things: the design of ash
script themselves can be improved for some speed-up. i.e: when you
don't use pattern, using fgrep instead of grep speed-up by 75%. The
ash build-in substitution is **a lot** faster than the sed one :
Message="SliTaz is great"
echo "${Message/great/GREAT!}"
is **>60x faster than** :
echo "SliTaz is great" | sed 's/great/GREAT!'
There's some other variable substitution which can be usefull in some
case.

Ok, that was some generic information instead of an answer about the
particular case you pointed...
I'm playing with libtazpkgbox to see if I'm able to improve speedup.

> And for core ISO, I propose to remove Transmission/libcurl and to use a
> web based bittorrent client such as: http://www.torrific.com/

I'm against replace one of the better bittorent apps by a commercial
on-line service... First, it's doesn't match at all the "spirit" of
P2P (where are peers, now???); then, check them "privacy" (non-privacy
in fact) policy at http://www.torrific.com/privacy/ (** comes from me) :

> 2. INFORMATION COLLECTION PRACTICES
> 2.1 TYPES OF INFORMATION COLLECTED
> (a) TRAFFIC DATA COLLECTED. We automatically track and collect the
> following categories of information when you visit our Site: (1) IP
> addresses; (2) domain servers; (3) types of computers accessing the
> Site; and (4) types of web browsers used to access the Site
> (collectively “Traffic Data”). Traffic Data is anonymous information
> that does not personally identify you but is helpful for **marketing**
> purposes or for improving your experience on the Site. We also use
> “cookies” to customize content specific to your interests, to ensure
> that you do not see the same **advertisement** repeatedly, and to store
> your password so you do not have to re-enter it each time you visit the
> Site.

Ok, so they save you're IP adress and use «"cookies" to customize
content specific to your interests [...]», at the same time they say
that «Traffic Data is anonymous information that does not personally
identify you»... Nothing strange ? And check the next chapter :

> (b) PERSONAL INFORMATION COLLECTED. In order for you to access certain
> **premium services** and to **purchase products** that we offer via the Site,
> we require you to provide us with certain information that personally
> **identifies you** (“Personal Information”). Personal Information includes
> the following categories of information: (1) Contact Data (such as your
> name, mailing address, and e-mail address); and (2) Demographic Data
> (such as your zip code, age, and income). Acceleration Labs does not
> collect or store Financial Data (such as your account or credit card
> number). If you subscribe to premium services, however, we do require
> you to provide Financial Data to a payment processing company subject
> to that vendor’s privacy policy. If you communicate with us by e-mail,
> post messages to any of our chat groups, bulletin boards, or forums, or
> otherwise complete online forms, surveys, or contest entries, **any
> information provided in such communication may be collected as Personal
> Information.**

Chapter d:
> (d) THIRD PARTY ADVERTISEMENTS. Advertisements and other content
> appearing on our site may be delivered (or "served") directly to you by
> third parties. These third parties automatically receive your IP
> address when this happens and may also download cookies to your
> computer, or use other technologies to measure the effectiveness of
> their ads and to personalize advertising content. We do not have access
> to or control of the cookies that may be placed by the third parties
> advertising or providing content to you through our site.
> 
> This privacy policy covers the use of cookies and Personal Information
> by us. **It does not cover third party use of cookies or other **tracking**
> technologies.**

etc... it's all about that, and at the end the famous:

> 4. UPDATES AND CHANGES TO PRIVACY POLICY.
> We reserve the right, at any time and without notice, to add to,
> change, update, or modify this Privacy Policy, simply by posting such
> change, update, or modification on the Site and **without any other
> notice to you**. Any such change, update, or modification will be
> effective immediately upon posting on the Site. 

All this means: all datas collected about you from our service is our
property and you have NO RIGHT concerning them... Well, do you really
want to push SliTaz users into this ? I don't think so...

-- 
GoKhlaYeh <gokhlayeh@xxxxxxxxxx>


---
SliTaz GNU/Linux Mailing list - http://www.slitaz.org/