Fastest way to compare directory state, or hashing for fun and profit

We have a PHP application, and were thinking it might be advantageous to have the application know if there was a change in its makeup since the last execution. Mainly due to managing caches and such, and knowing that our applications are sometimes accessed by people who don't remember to clear the cache on changes. (Changing the people is the obvious answer, but alas, not really achievable)

We've come up with this, which is the fastest we've managed to eke out, running an average 0.08 on a developer machine for a typical project. We've experimented with shasum,md5 and crc32, and this is the fastest. We are basically md5ing the contents of every file, and md5'ing that output. Security isnt a concern, we're just interested in detecting filesystem changes via a differing checksum.

time (find application/ -path '*/.svn' -prune -o -type f -print0 | xargs -0 md5 | md5)

I suppose the question is, can this be optimised any further?

(I realise that pruning svn will have a cost, but find takes the least amount of time out of the components, so it will be pretty minimal. We're testing this on a working copy atm)

Answers


You can be notified of filesystem modifications using the inotify extension.

It can be installed with pecl:

pecl install inotify

Or manually (download, phpize && ./configure && make && make install as usual).

This is a raw binding over the linux inotify syscalls, and is probably the fastest solution on linux.

See this example of a simple tail implementation: http://svn.php.net/viewvc/pecl/inotify/trunk/tail.php?revision=262896&view=markup


If you want a higher level library, or suppot for non-linux systems, take a look at Lurker.

It works on any system, and can use inotity when it's available.

See the example from the README:

$watcher = new ResourceWatcher;
$watcher->track('an arbitrary id', '/path/to/views');

$watcher->addListener('an arbitrary id', function (FilesystemEvent $event) {
    echo $event->getResource() . 'was' . $event->getTypeString();
});

$watcher->start();

Need Your Help

mprotect entire program, to run dangerous code

c linux posix sandbox mprotect

I have a small program that mmaps potentially dangerous executable code (with PROT_EXEC), calls prctl(PR_SET_SECCOMP, 1) and then executes this mmap'd code. This is all well and good, and allows me...

How does one change an instruction with a hex editor?

assembly hex reverse-engineering instructions ida

I am messing around with some reverse engineering, but I don't know how to change a jnz to jz with a hex editor. I understand it will vary between systems but I'm not sure where to look to find this

About UNIX Resources Network

Original, collect and organize Developers related documents, information and materials, contains jQuery, Html, CSS, MySQL, .NET, ASP.NET, SQL, objective-c, iPhone, Ruby on Rails, C, SQL Server, Ruby, Arrays, Regex, ASP.NET MVC, WPF, XML, Ajax, DataBase, and so on.