Python code parallelization for ALMA, challenges and alternatives

In the last few years development of Python code for science and data reduction purposes has gained significant popularity. ESO in itself uses a Python-based archiving  system for VLT and ALMA data. Also the data reduction suite for ALMA data is python-based. Rapid development is fostered by a big community and a wide range of already available packages. However Python enforces locking mechanisms, to ensure thread safety, that effectively reduce the capacity of Python to use only one core. In this context a number of alternatives have been developed by the community to emulate actual multi-threading and make parallel processing easier to use from Python, preserving interactivity.




CC BY 4.0