基于分布式执行框架的低频射电干涉阵列成像管线优化
作者:
作者单位:

1. 桂林电子科技大学信息与通信学院认知无线电与信息处理教育部重点实验室 桂林 541004;2. 中国科学院上海天文台 上海 200030;3. 云南大学物理与天文学院 昆明 650500

作者简介:

通讯作者:

中图分类号:

基金项目:

国家重点研发计划项目(2018YFA0404603)资助


Low-frequency Radio Interferometric Array Imaging Pipeline Optimization Based on Distributed Execution Framework
Author:
Affiliation:

1. Key Laboratory of Cognitive Radio and Information Processing of the Ministry of Education, School of Information and Communication, Guilin University of Electronic Science and Technology, Guilin 541004;2. Shanghai Astronomical Observatory, Chinese Academy of Sciences, Shanghai 200030;3. School of Physics and Astronomy, Yunnan University, Kunming 650500;

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    平方公里阵列(Square Kilometre Array, SKA)项目是建设全球最大射电望远镜的国际合作项目, 其灵敏度和测量速度将比当前所有的射电望远镜都要高出一个数量级. 连续谱巡天是SKA的主要观测模式之一, 基于连续谱成像建立巡天区域的标准星图, 将能为后续天文科学研究奠定重要基础. 银河系与河外星系全天默奇森宽场阵列拓展巡天(GaLactic and Extragalactic All-sky Murchison Widefield Array survey eXtended, GLEAM-X)是2018---2020年利用SKA先导望远镜默奇森宽场阵列(Murchison Wide-field Array, MWA)二期拓展阵列开展的新的射电连续谱巡天项目, 观测期间积累了大量的低频巡天观测数据. 海量观测数据的自动化、大批量处理是SKA望远镜项目所面临的的最大挑战和难题之一, 基于分布式执行框架的成像管线优化经验将有助于解决海量数据处理问题. 详细介绍了GLEAM-X成像管线并对其进行整合和改进, 在中国SKA区域中心原型机(China SKA Regional Centre Prototype, CSRC-P)上实现了多条管线并行处理, 使用GLEAM-X观测数据验证成像管线系统的部署和测试其正确性. 随后为了优化管线提高处理效率, 使用数据激活流图形引擎(Data Activated Liu Graph Engine, DALiuGE), 将成像管线集成入DALiuGE执行框架中实现了管线的自动化分布式并行处理. 通过性能测试与结果分析表明, 基于DALiuGE执行框架进行优化的成像管线相较于传统的并行方式具有更好的性能优势、更灵活的适配性和可扩展性, 可支持未来SKA第一阶段试运行期间的大规模连续谱成像实验.

    Abstract:

    The Square Kilometre Array (SKA) project is an international collaboration to build the world's largest radio telescope, whose sensitivity and measurement speed will be an order of magnitude higher than those of all current radio telescopes. Radio continuum survey is one of the main observation mode of the SKA, and the establishment of a standard map of the survey area based on continuum imaging will provide an important foundation for subsequent astronomical science. The GaLactic and Extragalactic All-sky Murchison Widefield Array survey eXtended (GLEAM-X) is a project of the SKA pilot telescope Murchison Widefield Array (MWA) in 2018---2020. GLEAM-X is a new radio continuum survey project to be carried out with the MWA Phase II expansion array in 2018---2020. The experience of optimizing the imaging pipeline based on the distributed execution framework will help to solve the problem of massive data processing. In this paper, we describe the process steps of GLEAM-X imaging pipeline, integrate and improve it, and realize parallel processing of multiple pipelines on the China SKA Regional Centre Prototype (CSRC-P), and verify the deployment and test the correctness of the imaging pipeline system using GLEAM-X observation data. The GLEAM-X observations were used to validate the deployment of the imaging pipeline system and test its correctness. Then, to optimize the pipelines and improve the processing efficiency, the Data Activated Liu Graph Engine (DALiuGE) was used to integrate the imaging pipelines into the DALiuGE execution framework to automate the distributed parallel processing of the pipelines. Performance tests and results analysis show that the optimized imaging pipeline based on the DALiuGE execution framework has better performance, more flexible adaptability and scalability than the traditional parallel approach, and can support future large-scale continuum imaging experiments during the first phase of SKA commissioning.

    参考文献
    相似文献
    引证文献
引用本文

韦耀杰,符杰林,劳保强.基于分布式执行框架的低频射电干涉阵列成像管线优化[J].天文学报,2023,64(6):67. WEI Yao-jie, FU Jie-lin, LAO Bao-qiang. Low-frequency Radio Interferometric Array Imaging Pipeline Optimization Based on Distributed Execution Framework[J]. Acta Astronomica Sinica,2023,64(6):67.

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2022-10-10
  • 最后修改日期:
  • 录用日期:
  • 在线发布日期: 2023-11-30
  • 出版日期: