dispy Releases up to version 2.9

[Giri]

Below is list of release announcements / short summary of changes of dispy up to version 2.9.

• dispy version 2.9 (2012-04-16)
  • dispy version 2.9 has been released. In this version locking primitives have been re-implemented so that 'yield' can be used after lock is acquired and before releasing it (earlier versions had differing rules about when to use yield and when not to). Now all locking methods should be invoked with 'yield'.
  • Changed suspend method to accept 'alarm_value' which is the value AsynCoro resumes a coro with if given timeout expires. Earlier versions always used None for alarm_value.
  • It is now possible to raise StopIteration (indicating termination of generator function, similar to return in regular methods) with a value. Thus 'raise StopIteration(v)' is equivalent to 'return v' in normal methods.
  • Added 'set_daemon' method to coroutines. AsynCoro scheduler waits for all non-daemon coroutines to terminate before exiting, similar to the behavior of threads. With this, user program can create coroutines and not worry about having to wait for coroutines to terminate (with 'join' method of scheduler, for example).
  • Yesterday when I announced plans to release version 3.0, I didn't plan on these (many) changes. I am still hoping to release 3.0 in the next couple of days, unless others point out any issues.
• Ready for stable release? (2012-04-14)
  • Just updated dispy/asyncoro with RLock primitive (similar to RLock in threading module) and set_daemon method to coroutines. With these features, asyncoro is very similar to threading module (but without some of the drawbacks of threads).
  • Unless someone finds any issues with asyncoro/dispy, I am thinking of releasing version 3.0 and marking it stable in the next couple of days.
• dispy version 2.8 (2012-04-10)
  • dispy version 2.8 has been released. This version implements Condition
    primitive as per Condition primitive in threading module (earlier versions
    differed slightly and incomplete). In addition, added AsynCoroThreadPool and
    AsynCoroDBCursor. AsynCoroThreadPool can be used to create a pool of threads
    to schedule synchronous tasks that can't be made asynchronous.
    AsynCoroDBCursor uses AsynCoroThreadPool to convert synchronous database
    cursor operations into asynchronous operations. This works with MySQLdb under
    Python 2.x and pymysql under Python 3.x.
• dispy version 2.7 (2012-04-04)
  • dispy version 2.7 has been released. The changes are mostly in asyncoro
    framework. Semaphore and Event primitives have been added to asyncoro and
    CoroLock and CoroCondition have been renamed to Lock and Condition,
    respectively, so they correspond to primitives in threading module. Coro also
    takes arguments as per threading.Thread, as well as positional and keyword
    arguments. The idea is to keep the interface to asyncoro as close to threading
    module as possible. Also added API documentation to asyncoro.html
• dispy version 2.6 (2012-03-25)
  • dispy version 2.6 has been released. This version fixes an issue with
    Windows 64-bit when I/O Completion Ports are used. Earlier versions implemented I/O
    polling/event processing in separate thread. In this version asyncoro
    scheduler itself processes I/O events. As coroutines are supposed to use
    'yield' for any time consuming activity, such as I/O completion or coro.sleep
    this works as expected. However, users (developers using asyncoro) must be
    careful not to call time.sleep etc. from coroutines.
• dispy version 2.5 (2012-03-22)
  • dispy version 2.5 has been released. Previos versions had an issue with
    polling mechanisms causing flood of hangup events (such as when remote closed
    connection, or socket is not yet connected). This version fixes it.
• dispy version 2.4 (2012-03-16)
  dispy version 2.4 has been released. This version fixes a few issues with
  version 2.3:
  
  • If used to compute programs (instead of python functions), dispynode executes
    transferred program with full path.
  • 'poll' polling object takes timeouts as milliseconds, so timeouts are
    multiplied by 1000 for timeouts in seconds.
  • When dispy is used as a tool, the output, error etc., for the computations are
    printed after they are completed.
• dispy version 2.3 (2012-03-10)
  • dispy version 2.3 has been released. This version supports synchronous SSL
    (although most of dispy uses asynchronous sockets, there are couple of places
    where synchronous sockets are used). dispy uses messages that are tagged with
    length of the message so receiving socket knows how much data to read. This
    simplifies communication. The methods that implement these have been renamed
    from read/write/read_msg/write_msg to recvall/sendall/recv_msg/send_msg to
    avoid confusion with standard read/write methods.
• dispy version 2.2 (2012-03-06)
  • dispy version 2.2 has been released. This version fixes asyncoro so it
    terminates cleanly when its terminate method is called.
• dispy version 2.1 (2012-03-04)
  • dispy version 2.1 has been released. This version adds support for Windows
    I/O Completion Ports (IOCP) for high performance and scalability. To use IOCP,
    pywin32 (http://pywin32.sf.net) must be installed.
• dispy version 2.0 (2012-03-01)
  • dispy version 2.0 has been released. This is based on discopy
    implementation with asynchronous sockets and coroutines. The old threaded implementation is
    not needed and there won't be discopy releases anymore.
• discopy version 1.8 (2012-02-27)
  • discopy (an implementation of dispy with asynchronous sockets and
    coroutines) version 1.8 has been released. This version supports Windows (first version of
    discopy to work with Windows). It also fixes issues with fault recovery.
    
    If others confirm that discopy works with Windows, I will move discopy code to
    dispy and release future versions based on current discopy (i.e., with
    asynchronous sockets and coroutines).
• discopy version 1.7 (2012-02-25)
  • discopy (implementation of dispy with asynchronous sockets and coroutines)
    version 1.7 has been released. Earlier versions of discopy checked for socket
    timeouts every 10 seconds. Thus, timeout exceptions are thrown up to 10
    seconds after it is supposed to be timed out. Version 1.7 implements timeouts
    properly.
• discopy version 1.6 (2012-02-24)
  • discopy version 1.6 has been released. Although previous version was 0.6,
    this huge jump in number doesn't signify much, other than a few fixes and cleanup
    to asyncoro. However, now both dispy and discopy are at same version, to
    convey that discopy is not far behind (on the contrary, discopy has received
    more attention than dispy).
    
    discopy has been tested with Linux and OS X, so it is the default download for
    all platforms except Windows. Until someone confirms status of discopy with
    Windows, I will mark dispy default download for Windows.
• discopy version 0.6 (2012-02-22)
  discopy version 0.6 has been released. The major changes since version 0.5
  are:
  
  • Exception handling and propagation have been fixed in asyncoro
  • Option 'resubmit' has been renamed to 'reentrant' as latter is more
    appropriate
• discopy version 0.5 (2012-02-19)
  • discopy (implementation of dispy with asynchronous sockets and coroutines)
    version 0.5 has been released. The major changes since version 0.4 are: SSL is
    now supported, asyncoro has been cleaned up (e.g., no reason to pass 'coro'
    for locking methods, coros can be selectively terminated), and, as usual, bug
    fixes. I don't expect anymore active development in either dispy or discopy.
    The only pending issue with discopy is confirmation that it does (not) work
    with Windows. Once this is handled, I am thinking of making discopy default
    download, as discopy will scale from one node to many, many nodes without the
    overheads of threads.
• dispy version 1.5 (2012-02-15)
  • dispy version 1.5 has been released. This fixes an issue in dispynode which
    ignored pulse timeout (if used) for computations.
• discopy version 0.4 (2012-02-15)
  • discopy (an implementation of dispy with asynchronous I/O and coroutines)
    version 0.4 has been released. Changes since 0.3 are cleanups, especially in
    asyncoro.py, and bug fixes.
    
    Last few days have seen frequent updates to discopy and releases on successive
    days. Now the framework is mostly stable, so development and releases
    hopefully are not hectic.
• discopy version 0.3 (2012-02-14)
  • discopy (an implementation of dispy with asynchronous I/O and coroutines) has
    been released. Other than a few bug fixes and cleanups, the major change is
    that the asynchronous I/O and coroutine framework used in dispy is made
    independent (as asyncoro.py). Others could use this framework for similar
    projects independent of dispy.
• dispy version 1.3 (2012-02-13)
  • Version 1.4 of dispy has been released. Changes since version 1.2 are
    mostly bug fixes. Note that there was a testing release of version 1.3 that has been
    pulled and released as discopy-0.1, to avoid confusion with dispy releases.
• discopy version 0.2 (2012-02-13)
  • discopy (an implementation of dispy using asynchronous I/O and coroutines)
    version 0.2 has been released. The changes since version 0.1 are bug fixes.
• discopy version 0.1 (2012-02-12)
  • What was earlier released as version 1.3 (testing) has been fixed up and
    released as 'discopy-0.1', to avoid confusion with dispy. discopy is not
    tested thoroughly; it is not tested on Windows at all. If you find issues, or
    find it working with Windows, post follow up.
    
    Whereas dispy is implemented with threads to process concurrent connections,
    discopy is implemented using asynchronous I/O and coroutines (without
    threads). There are pros and cons to both approaches. The hope is that discopy
    should be able to scale well for large number of connections, but at this
    point this is moot - I think this exercise is a bit academic until someone
    finds it better suited to them.
• dispy version 1.3 (2012-02-08)
  • There is a new release in 'testing', version 1.3. This uses asynchronous
    I/O, coroutines instead of threads to handle multiple connections. Asynchronous I/O
    is meant for scaling dispy to handle large number of processors. Coroutines
    are apparently not as efficient as threads, but don't have some drawbacks of
    threads (in Python). I am not sure if at some point coroutines will be
    replaced with threads.
    
    I have tested 1.3 minimally with Linux and OS X. It "should" work with
    Windows, but needs testing. If you use it Windows, let me know if it works or
    not. Once it is tested throughly, I will release 1.4 as normal (non-testing)
    release.
• dispy version 1.2 (2012-01-27)
  Version 1.2 has been released. Major changes from version 1.1 are:
  
  • SharedJobCluster supports zombie client detection, fault recovery of jobs
  • dispynode and SharedJobCluster recover from transient network failures (client
    is not reachable intermittently)
• dispy version 1.1 (2012-01-21)
  Version 1.1 has been released. Major changes from 1.0 are:
  
  • Added client side fault recovery (with 'fault_recover' option). Note that
  • dispy already supports server side (node) fault recovery with 'resubmit=True'
    option.
  • SharedJobCluster now supports transferring files.
  • dispynode supports zombie detection: If a dispy client dies unexpectedly, the
    node can free itself from that client so other clients can use the node, if
    zombie_interval (minutes) is set.
• dispy version 1.0 (2012-01-08)
  dispy version 1.0 has been released. Major changes since last release are:
  
  • Job's 'state' field has been changed to 'status'. Once a job is created (with
    cluster.submit), its ultimate state is one of Finished (job's execution
    finished with result of computation in 'result' field), Terminated (either job
    raised an exception during execution or it has been cancelled after it has
    been scheduled for execution on a node) or Cancelled (job has been cancelled
    before it was scheduled for execution, and thus doesn't require any cleanup).
  • Bug fixes
• dispy version 0.7 (2011-12-31)
• Version 0.7 is released. Major changes are:
  • After jobs are scheduled with callbacks, calling cluster.wait() will guarantee
    that all callbacks are called before wait returns.
  • In some computations, such as optimizations, it may be useful to send
    iterative results back to the client (which can, for example, terminate the
    computations if the best result is acceptable). Such computations (only if
    they are Python functions) can now use 'dispy_provisional_result(result)',
    where 'result' is whatever Python (serializable) object. For such provisional
    results, callbacks are called if cluster is created with 'callback' argument.
  • The callbacks can use job.state to distinguish if result is final
    (DispyJob.Finished) or provisional (DispyJob.ProvisionalResult).
• dispy version 0.6 (2011-12-29)
  Version 0.6 is released. Major changes are:
  • Added 'callback' feature so dispy can call provided function when a job is
    finished executing (when job's results are available)
  • Added option to send 'ping' interval to force dispy send messages to discover
    new nodes
  • Fixed dispy so if a node dies while serving job requests and then comes back
    up, dispy will recognize and resuse that node for further job requests
  • Improved reliability.
• dispy version 0.4 (2011-12-08)
  • Version 0.4 has been released with preliminary support for Windows. Thanks
    to Matt Hutchinson for testing and giving feedback.
• dispy version 0.3 (2011-12-04)
  • Version 0.3 has been released. The changes since 0.2 have been fixes since
    first 0.2 release - I have been updating 0.2 release with fixes, so you may
    not see many change since latest 0.2 release. We now also have svn, so going
    forward it will be easy to see updates/fixes.
    
    I still don't know if dispy works with Windows; if you try it, please confirm
    if it works or not.
    
    It would help further development if you can post feedback (e.g., if you found
    it useful, how you use it, any shortcomings in features etc.).
• dispy version 0.2 (2011-09-30)
  • Hello
    
    Version 0.2 fixed a few issues. It is expected to work with Windows too,
    although it is not confirmed. If you use it with Windows, please confirm if it
    works (or not).
    
    Giri