Ubuntu QA:
BlogBrainstormPackage status
Log in
Ubuntu QA
The Ubuntu community has contributed 21986 ideas, 135057 comments, 2615221 votes
Idea sandbox Idea sandbox
Popular ideas Popular ideas
Ideas in development Ideas in development
Implemented ideas Implemented ideas
Idea #9609: Store dpkg and apt files in an SQLite database

Written by flammon the 7 Jun 08 at 01:34. Category: System. Related project: Nothing/Others. Status: New
Rationale
Store the dpkg available and status files in an SQLite database to improve performance, reduce file size, increase integrity, have better flexibility when searching and simplify updates.

The proof of concept has already started.

http://people.debian.org/~seanius/dpkg-sqlite/

sean finney wrote:

I have checked it out, very nice, besides missing a -ldl flag, everything went
according to plan. The speed is simply amazing, here is a small benchmark I did
on my machine:

omega3:/home/joe/build/dpkg-sqlite# echo 1 > /proc/sys/vm/drop_caches
omega3:/home/joe/build/dpkg-sqlite# time dpkg -S /usr/bin/gcc
gcc: /usr/bin/gcc

real 0m4.778s
user 0m0.252s
sys 0m0.152s
omega3:/home/joe/build/dpkg-sqlite# echo 1 > /proc/sys/vm/drop_caches
omega3:/home/joe/build/dpkg-sqlite# time ./poc /usr/bin/gcc
querying as if listing package /usr/bin/gcc:
querying as if for owner of file /usr/bin/gcc:
gcc: /usr/bin/gcc

real 0m1.306s
user 0m0.240s
sys 0m0.096s
omega3:/home/joe/build/dpkg-sqlite#

The difference is drastic. One of the main concerns now is reliability. Does
anyone here know of any SQLite horror stories they might like to share ?

Tags: (none)

75
votes
up equal down
Solution #1: Auto-generated solution of idea #9609
Written by flammon the 7 Jun 08 at 01:34.
Ubuntu Brainstorm was updated in January 2009. Since the idea #9609 was submitted before this update, its rationale and solution are not separated. Please vote accordingly, and if you have the necessary rights, please separate the rationale from the solution. Thanks!

Propose your solution

Attachments
No attachments.


Duplicates


Comments
flammon wrote on the 7 Jun 08 at 02:12
I forgot to mention, SQLite also supports triggers which could be more robust than the current /var/lib/dpkg/triggers method and it would allow parallelization of tasks.

Triggers would be fired based on package dependencies. Packages with no dependacies would be installed first in parallel and they would fire triggers to continue the installation of their dependents.

Auzy wrote on the 7 Jun 08 at 06:03
This is a great idea, and it doesn't introduce any backward compatibility issues.

And it makes perfect sense :D

SQL reduces the need to reinvent the wheel. Because they can handle all the optimisation of the database and the code. We just worry about using it.

And, SQL has more features. We get the new features free.

Awesome idea. +1

Sector wrote on the 7 Jun 08 at 10:21
I completely agree. I use SQLite in programs I make all the time, its simple, fast and just works. Although I don't really get why the firefox people took so long to make a bookmarking system in SQLite :).

I think application managing systems in a distro, which are responsible for updates/new programs, should be at the pinnacle of their speed capacity, because it's a thing that many people often come in contact with

+1

sebsauvage wrote on the 9 Jun 08 at 11:14
+1

SQLite is very fast, space-efficient and ensures data integrity.

But woudln't the fsync() problem (as in Firefox 3) cause any performance problem ?
(Shouldn't since dpkg is not the kind of program you run all the time.)

Auzy wrote on the 9 Jun 08 at 15:12
The fsync problem I think they said was due to windows not properly syncing with the disk. I might be wrong though.

BagOfMostlyWater wrote on the 26 Dec 08 at 07:44
Should the next step be to file a bug in Debian's bug tracker against dpkg? Ian Jackson believes this is a bad idea apparently:

http://www.mail-archive.com/debian-dpkg@lists.debian.org/msg11566.html

kennydude wrote on the 15 Jan 09 at 18:45
This would be a excellent idea. Come on dev's why don't you get round to doing this already!

yup wrote on the 21 Jan 10 at 14:55
It is a pity that there is no progress on this :( ...


Post your comment