Paste: "real world" parallel map

Author: kobi
Mode: factor
Date: Fri, 17 Sep 2010 13:11:28
Plain Text | 
! parallel combinators run out of memory because they prepare all threads beforehand.
! here we seperate to groups, run a sequential map on the groups, but inside the groups we use parallel combinator.

! this code can be adapted for parallel-each as well.


: (limited-parallel) ( seq quot limit -- seq quot' )    
    tuck [ <groups> ] 2dip
    <semaphore> swap '[ _ _ with-semaphore ] ; inline
    
: limited-parallel-map ( seq quot limit -- seq' )
    (limited-parallel) '[ _ parallel-map ] map ; inline    


! untested:
: limited-parallel-each ( seq quot limit -- )
    (limited-parallel) '[ _ parallel-each ] each ; inline

New Annotation

Summary:
Author:
Mode:
Body: