Eintrag weiter verarbeiten
Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN
Gespeichert in:
| Zeitschriftentitel: | Journal of Physics: Conference Series |
|---|---|
| Personen und Körperschaften: | , , |
| In: | Journal of Physics: Conference Series, 1525, 2020, 1, S. 012028 |
| Format: | E-Article |
| Sprache: | Unbestimmt |
| veröffentlicht: |
IOP Publishing
|
| Schlagwörter: |
| author_facet |
Abed Abud, Adam Le Goff, Fabrice Avolio, Giuseppe Abed Abud, Adam Le Goff, Fabrice Avolio, Giuseppe |
|---|---|
| author |
Abed Abud, Adam Le Goff, Fabrice Avolio, Giuseppe |
| spellingShingle |
Abed Abud, Adam Le Goff, Fabrice Avolio, Giuseppe Journal of Physics: Conference Series Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN General Physics and Astronomy |
| author_sort |
abed abud, adam |
| spelling |
Abed Abud, Adam Le Goff, Fabrice Avolio, Giuseppe 1742-6588 1742-6596 IOP Publishing General Physics and Astronomy http://dx.doi.org/10.1088/1742-6596/1525/1/012028 <jats:title>Abstract</jats:title> <jats:p>Over the next few years, the LHC will prepare for the upcoming High-Luminosity upgrade in which it is expected to deliver ten times more pp collisions. This will create a harsher radiation environment and higher detector occupancy. In this context, the ATLAS experiment, one of the general purpose experiments at the LHC, plans substantial upgrades to the detectors and to the trigger system in order to efficiently select events. Similarly, the Data Acquisition System (DAQ) will have to redesign the data-flow architecture to accommodate for the large increase in event and data rates. The Phase-II DAQ design involves a large distributed storage system that buffers data read out from the detector, while a computing farm (Event Filter) analyzes and selects the most interesting events. This system will have to handle 5.2 TB/s of input data for an event rate of 1 MHz and provide access to 3 TB/s of these data to the filtering farm. A possible implementation for such a design is based on distributed file systems (DFS) which are becoming ubiquitous among the big data industry. Features of DFS such as replication strategies and smart placement policies match the distributed nature and the requirements of the new data-flow system. This paper presents an up-to-date performance evaluation of some of the DFS currently available: GlusterFS, HadoopFS and CephFS. After characterization of the future data-flow systems workload, we report on small-scale raw performance and scalability studies. Finally, we conclude on the suitability of such systems to the tight constraints expected for the ATLAS experiment in phase-II and, in general, what benefits the HEP community can take from these storage technologies.</jats:p> Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN Journal of Physics: Conference Series |
| doi_str_mv |
10.1088/1742-6596/1525/1/012028 |
| facet_avail |
Online Free |
| format |
ElectronicArticle |
| fullrecord |
blob:ai-49-aHR0cDovL2R4LmRvaS5vcmcvMTAuMTA4OC8xNzQyLTY1OTYvMTUyNS8xLzAxMjAyOA |
| id |
ai-49-aHR0cDovL2R4LmRvaS5vcmcvMTAuMTA4OC8xNzQyLTY1OTYvMTUyNS8xLzAxMjAyOA |
| institution |
DE-105 DE-14 DE-Ch1 DE-L229 DE-D275 DE-Bn3 DE-Brt1 DE-D161 DE-Zwi2 DE-Gla1 DE-Zi4 DE-15 DE-Pl11 DE-Rs1 |
| imprint |
IOP Publishing, 2020 |
| imprint_str_mv |
IOP Publishing, 2020 |
| issn |
1742-6588 1742-6596 |
| issn_str_mv |
1742-6588 1742-6596 |
| language |
Undetermined |
| mega_collection |
IOP Publishing (CrossRef) |
| match_str |
abedabud2020performanceevaluationofdistributedfilesystemsforthephaseiiupgradeoftheatlasexperimentatcern |
| publishDateSort |
2020 |
| publisher |
IOP Publishing |
| recordtype |
ai |
| record_format |
ai |
| series |
Journal of Physics: Conference Series |
| source_id |
49 |
| title |
Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN |
| title_unstemmed |
Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN |
| title_full |
Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN |
| title_fullStr |
Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN |
| title_full_unstemmed |
Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN |
| title_short |
Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN |
| title_sort |
performance evaluation of distributed file systems for the phase-ii upgrade of the atlas experiment at cern |
| topic |
General Physics and Astronomy |
| url |
http://dx.doi.org/10.1088/1742-6596/1525/1/012028 |
| publishDate |
2020 |
| physical |
012028 |
| description |
<jats:title>Abstract</jats:title>
<jats:p>Over the next few years, the LHC will prepare for the upcoming High-Luminosity upgrade in which it is expected to deliver ten times more pp collisions. This will create a harsher radiation environment and higher detector occupancy. In this context, the ATLAS experiment, one of the general purpose experiments at the LHC, plans substantial upgrades to the detectors and to the trigger system in order to efficiently select events. Similarly, the Data Acquisition System (DAQ) will have to redesign the data-flow architecture to accommodate for the large increase in event and data rates. The Phase-II DAQ design involves a large distributed storage system that buffers data read out from the detector, while a computing farm (Event Filter) analyzes and selects the most interesting events. This system will have to handle 5.2 TB/s of input data for an event rate of 1 MHz and provide access to 3 TB/s of these data to the filtering farm. A possible implementation for such a design is based on distributed file systems (DFS) which are becoming ubiquitous among the big data industry. Features of DFS such as replication strategies and smart placement policies match the distributed nature and the requirements of the new data-flow system. This paper presents an up-to-date performance evaluation of some of the DFS currently available: GlusterFS, HadoopFS and CephFS. After characterization of the future data-flow systems workload, we report on small-scale raw performance and scalability studies. Finally, we conclude on the suitability of such systems to the tight constraints expected for the ATLAS experiment in phase-II and, in general, what benefits the HEP community can take from these storage technologies.</jats:p> |
| container_issue |
1 |
| container_start_page |
0 |
| container_title |
Journal of Physics: Conference Series |
| container_volume |
1525 |
| format_de105 |
Article, E-Article |
| format_de14 |
Article, E-Article |
| format_de15 |
Article, E-Article |
| format_de520 |
Article, E-Article |
| format_de540 |
Article, E-Article |
| format_dech1 |
Article, E-Article |
| format_ded117 |
Article, E-Article |
| format_degla1 |
E-Article |
| format_del152 |
Buch |
| format_del189 |
Article, E-Article |
| format_dezi4 |
Article |
| format_dezwi2 |
Article, E-Article |
| format_finc |
Article, E-Article |
| format_nrw |
Article, E-Article |
| _version_ |
1792321397095137296 |
| geogr_code |
not assigned |
| last_indexed |
2024-03-01T11:01:21.529Z |
| geogr_code_person |
not assigned |
| openURL |
url_ver=Z39.88-2004&ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fvufind.svn.sourceforge.net%3Agenerator&rft.title=Performance+evaluation+of+distributed+file+systems+for+the+phase-II+upgrade+of+the+ATLAS+experiment+at+CERN&rft.date=2020-04-01&genre=article&issn=1742-6596&volume=1525&issue=1&pages=012028&jtitle=Journal+of+Physics%3A+Conference+Series&atitle=Performance+evaluation+of+distributed+file+systems+for+the+phase-II+upgrade+of+the+ATLAS+experiment+at+CERN&aulast=Avolio&aufirst=Giuseppe&rft_id=info%3Adoi%2F10.1088%2F1742-6596%2F1525%2F1%2F012028&rft.language%5B0%5D=und |
| SOLR | |
| _version_ | 1792321397095137296 |
| author | Abed Abud, Adam, Le Goff, Fabrice, Avolio, Giuseppe |
| author_facet | Abed Abud, Adam, Le Goff, Fabrice, Avolio, Giuseppe, Abed Abud, Adam, Le Goff, Fabrice, Avolio, Giuseppe |
| author_sort | abed abud, adam |
| container_issue | 1 |
| container_start_page | 0 |
| container_title | Journal of Physics: Conference Series |
| container_volume | 1525 |
| description | <jats:title>Abstract</jats:title> <jats:p>Over the next few years, the LHC will prepare for the upcoming High-Luminosity upgrade in which it is expected to deliver ten times more pp collisions. This will create a harsher radiation environment and higher detector occupancy. In this context, the ATLAS experiment, one of the general purpose experiments at the LHC, plans substantial upgrades to the detectors and to the trigger system in order to efficiently select events. Similarly, the Data Acquisition System (DAQ) will have to redesign the data-flow architecture to accommodate for the large increase in event and data rates. The Phase-II DAQ design involves a large distributed storage system that buffers data read out from the detector, while a computing farm (Event Filter) analyzes and selects the most interesting events. This system will have to handle 5.2 TB/s of input data for an event rate of 1 MHz and provide access to 3 TB/s of these data to the filtering farm. A possible implementation for such a design is based on distributed file systems (DFS) which are becoming ubiquitous among the big data industry. Features of DFS such as replication strategies and smart placement policies match the distributed nature and the requirements of the new data-flow system. This paper presents an up-to-date performance evaluation of some of the DFS currently available: GlusterFS, HadoopFS and CephFS. After characterization of the future data-flow systems workload, we report on small-scale raw performance and scalability studies. Finally, we conclude on the suitability of such systems to the tight constraints expected for the ATLAS experiment in phase-II and, in general, what benefits the HEP community can take from these storage technologies.</jats:p> |
| doi_str_mv | 10.1088/1742-6596/1525/1/012028 |
| facet_avail | Online, Free |
| format | ElectronicArticle |
| format_de105 | Article, E-Article |
| format_de14 | Article, E-Article |
| format_de15 | Article, E-Article |
| format_de520 | Article, E-Article |
| format_de540 | Article, E-Article |
| format_dech1 | Article, E-Article |
| format_ded117 | Article, E-Article |
| format_degla1 | E-Article |
| format_del152 | Buch |
| format_del189 | Article, E-Article |
| format_dezi4 | Article |
| format_dezwi2 | Article, E-Article |
| format_finc | Article, E-Article |
| format_nrw | Article, E-Article |
| geogr_code | not assigned |
| geogr_code_person | not assigned |
| id | ai-49-aHR0cDovL2R4LmRvaS5vcmcvMTAuMTA4OC8xNzQyLTY1OTYvMTUyNS8xLzAxMjAyOA |
| imprint | IOP Publishing, 2020 |
| imprint_str_mv | IOP Publishing, 2020 |
| institution | DE-105, DE-14, DE-Ch1, DE-L229, DE-D275, DE-Bn3, DE-Brt1, DE-D161, DE-Zwi2, DE-Gla1, DE-Zi4, DE-15, DE-Pl11, DE-Rs1 |
| issn | 1742-6588, 1742-6596 |
| issn_str_mv | 1742-6588, 1742-6596 |
| language | Undetermined |
| last_indexed | 2024-03-01T11:01:21.529Z |
| match_str | abedabud2020performanceevaluationofdistributedfilesystemsforthephaseiiupgradeoftheatlasexperimentatcern |
| mega_collection | IOP Publishing (CrossRef) |
| physical | 012028 |
| publishDate | 2020 |
| publishDateSort | 2020 |
| publisher | IOP Publishing |
| record_format | ai |
| recordtype | ai |
| series | Journal of Physics: Conference Series |
| source_id | 49 |
| spelling | Abed Abud, Adam Le Goff, Fabrice Avolio, Giuseppe 1742-6588 1742-6596 IOP Publishing General Physics and Astronomy http://dx.doi.org/10.1088/1742-6596/1525/1/012028 <jats:title>Abstract</jats:title> <jats:p>Over the next few years, the LHC will prepare for the upcoming High-Luminosity upgrade in which it is expected to deliver ten times more pp collisions. This will create a harsher radiation environment and higher detector occupancy. In this context, the ATLAS experiment, one of the general purpose experiments at the LHC, plans substantial upgrades to the detectors and to the trigger system in order to efficiently select events. Similarly, the Data Acquisition System (DAQ) will have to redesign the data-flow architecture to accommodate for the large increase in event and data rates. The Phase-II DAQ design involves a large distributed storage system that buffers data read out from the detector, while a computing farm (Event Filter) analyzes and selects the most interesting events. This system will have to handle 5.2 TB/s of input data for an event rate of 1 MHz and provide access to 3 TB/s of these data to the filtering farm. A possible implementation for such a design is based on distributed file systems (DFS) which are becoming ubiquitous among the big data industry. Features of DFS such as replication strategies and smart placement policies match the distributed nature and the requirements of the new data-flow system. This paper presents an up-to-date performance evaluation of some of the DFS currently available: GlusterFS, HadoopFS and CephFS. After characterization of the future data-flow systems workload, we report on small-scale raw performance and scalability studies. Finally, we conclude on the suitability of such systems to the tight constraints expected for the ATLAS experiment in phase-II and, in general, what benefits the HEP community can take from these storage technologies.</jats:p> Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN Journal of Physics: Conference Series |
| spellingShingle | Abed Abud, Adam, Le Goff, Fabrice, Avolio, Giuseppe, Journal of Physics: Conference Series, Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN, General Physics and Astronomy |
| title | Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN |
| title_full | Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN |
| title_fullStr | Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN |
| title_full_unstemmed | Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN |
| title_short | Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN |
| title_sort | performance evaluation of distributed file systems for the phase-ii upgrade of the atlas experiment at cern |
| title_unstemmed | Performance evaluation of distributed file systems for the phase-II upgrade of the ATLAS experiment at CERN |
| topic | General Physics and Astronomy |
| url | http://dx.doi.org/10.1088/1742-6596/1525/1/012028 |