Asynchronous file writing possible in python?

Is there an existing module or an easy way to use aio_write to write to a file asynchronously in python? I know by file io that comes with Python is all blocking which is fine in most cases, but for this particular case I need writes not to block the application at all, or at least as minimally as possible.

Answers


Twisted has non-blocking writes on file descriptors. If you're writing async code, I'd expect you to be using twisted, anyway. :)


As I understand things, asynchronous I/O is not quite the same as non-blocking I/O.

In the case of non-blocking I/O, once a file descriptor is setup to be "non-blocking", a read() system call (for instance) will return EWOULDBLOCK (or EAGAIN) if the read operation would block the calling process in order to complete the operation. The system calls select(), poll(), epoll(), etc. are provided so that the process can ask the OS to be told when one or more file descriptors become available for performing some I/O operation.

Asynchronous I/O operates by queuing a request for I/O to the file descriptor, tracked independently of the calling process. For a file descriptor that supports asynchronous I/O (raw disk devcies typically), a process can call aio_read() (for instance) to request a number of bytes be read from the file descriptor. The system call returns immediately, whether or not the I/O has completed. Some time later, the process then polls the operating system for the completion of the I/O (that is, buffer is filled with data).

A process (single-threaded) that only performs non-blocking I/O will be able to read or write from one file descriptor that is ready for I/O when another is not ready. But the process must still synchronously issue the system calls to perform the I/O to all the ready file descriptors. Whereas, in the asynchronous I/O case, the process is just checking for the completion of the I/O (buffer filled with data). With asynchronous I/O, the OS has the freedom to operate in parallel as much as possible to service the I/O, if it so chooses.

With that, are there any wrappers for the POSIX aio_read/write etc. system calls for Python?


Need Your Help

Error 3021 when deleting item from recordset

vba ms-access access-vba

When I use the code below, I sometimes receive a Error 3021. This only happens when I have one record in the recordset. Can you please tell me why, and how to fix it? It seems I've tried everything!

Updating record with Linq

c# asp.net sql database linq

I have a question regarding Linq and updating a record in the database. I don't know how to update a record when the fields that you use are not known. It is hard for me to explain this so I use the

About UNIX Resources Network

Original, collect and organize Developers related documents, information and materials, contains jQuery, Html, CSS, MySQL, .NET, ASP.NET, SQL, objective-c, iPhone, Ruby on Rails, C, SQL Server, Ruby, Arrays, Regex, ASP.NET MVC, WPF, XML, Ajax, DataBase, and so on.