GlPortal

Welcome to the GlPortal issue tracker. New to the project? Try our simple tasks.

  • Status New
  • Percent Complete
    0%
  • Task Type Feature Request
  • Category Backend / Core
  • Assigned To No-one
  • Operating System All
  • Severity Low
  • Priority Medium
  • Reported Version Development
  • Due in Version backlog
  • Due Date Undecided
  • Votes
  • Private
Attached to Project: GlPortal
Opened by Henry Hirsch - 18.03.2016
Last edited by Henry Hirsch - 19.03.2016

FS#128 - Asynchronous file I/O

https://bugzilla.libsdl.org/show_bug.cgi?id=2419

Currently, file RWops are blocking and synchronous (just like what's provided by the C FILE* and C++ fstream). This is all fine in good for non-interactive programs and initialization-time loading. But most applications of SDL are interactive, and many are games and media players, which need to read from the harddisk while remaining interactive.

Because file RWops are blocking (take control away from the main thread), reading too much data from the hard disk on the main thread makes programs non-interactive. This leads to messy solutions like creating another thread just to load data ( "Main thread renders loading animation while background thread uploads whole level along with textures. In fact I did notice that this takes slightly longer than doing everything in main but user experience is much better with main thread still operational, showing anims and gameplay tips. ": http://forums.libsdl.org/viewtopic.php?t=9897 ).

So, I request the addition of asynchronous I/O features to SDL.

POSIX non-blocking I/O is simple: pass the flag O_NONBLOCK to open(), and all calls to read/write will return without waiting for the disk (the return value will be the number of bytes written). Asynchronous file I/O was added to POSIX in 2001. Most versions of the interface do not work on sockets. See: http://pubs.opengroup.org/onlinepubs/009604599/basedefs/aio.h.html Non-blocking I/O can be multiplexed using select() in order to reduce the number of read()/write() syscalls: http://pubs.opengroup.org/onlinepubs/7908799/xsh/select.html With non-blocking I/O, SDL will have to maintain the total number of bytes written in order to determine when an I/O operation has completed.

Windows does not support synchronous (in order) non-blocking I/O. Instead it only supports asynchronous (out-of-order, or in Windows terms, overlapped) I/O. This is implemented by passing FILE_FLAG_OVERLAPPED to CreateFile and maintaining an OVERLAPPED structure for each I/O operation. See: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365683%28v=vs.85%29.aspx There are several ways to get indications of I/O completions using asynchronous I/O. One is to associate a kernel event with the OVERLAPPED structure passed to ReadFile/WriteFile. Another is to call GetOverlappedResult. But because an application that has real use of asynchronous I/O is probably performing operations on multiple files, it would be better to reduce the number of syscalls. This means using Read/WriteFileEx and entering an alertable wait state in the main loop (a new API function, SDL_PumpIO, for other threads; and probably inside SDL_PumpEvents) to execute I/O completion handlers. See: http://msdn.microsoft.com/en-us/library/windows/desktop/aa363772%28v=vs.85%29.aspx

It is entirely possible to implement this using the existing SDL_RWops interface. But an asynchronous SDL_RWop would break old code using SDL_RWops by its nature.

Instead, I recommend an additional API just for asynchronous file ops. While this could certainly be done as a third party library, I feel that it would be a better fit for the main library, using the SDL event queue to report completed/failed async ops with a new event.

Dorian Wouters commented on 30.03.2016 17:35

Actually this was planned since long ago, just not written down. The basic target was to have the next level's resources be loaded while you're playing the current one, a thing for which asynchronous I/O is required -- although initially supposed to be done with traditional blocking I/O in worker threads.

Date User Effort (H:M)

Loading...

Available keyboard shortcuts

Task List

Task Details

Add/Edit Task

TODO:complete the list
for accesskey usage different shortcuts on Windows, Mac, Linux .., currently shown for Firefox