# Download the last version on
# on /eos/user/s/sterbini/MD_ANALYSIS/cl2pdExamplePy3.ipynb
# See the details at https://github.com/sterbini/cl2pd
# download it from SWAN terminal by
# pip install --user git+https://github.com/sterbini/cl2pd.git
import cl2pd
from cl2pd import importData
pd = importData . pd #is the pandas package
cals = importData . cals #is the ldb variable of pytimber package
! pwd
/eos/home-s/sterbini/MD_ANALYSIS
From CALS to pandas DataFrame
We use the precious package from R. De Maria et al. (pytimber , https://github.com/rdemaria/pytimber ) to wrap its output in pandas DataFrames.
cals2pd function
This is the most important function of the importData class.
You can use different timezone, in this example we use Central European Time (local time at CERN).
variables = [ 'LHC.BCTDC.A6R4.B1:BEAM_INTENSITY' , 'LHC.BCTDC.A6R4.B2:BEAM_INTENSITY' ]
startTime = pd . Timestamp ( '2017-10-01 17:30' , tz = 'CET' )
endTime = pd . Timestamp ( '2017-10-01 17:31' , tz = 'CET' )
raw_data = importData . cals2pd ( variables , startTime , endTime )
raw_data . head ()
LHC.BCTDC.A6R4.B1:BEAM_INTENSITY
LHC.BCTDC.A6R4.B2:BEAM_INTENSITY
2017-10-01 15:30:00+00:00
1.470000e+09
270000000.0
2017-10-01 15:30:01+00:00
1.810000e+09
310000000.0
2017-10-01 15:30:02+00:00
1.690000e+09
-310000000.0
2017-10-01 15:30:03+00:00
1.990000e+09
-720000000.0
2017-10-01 15:30:04+00:00
1.270000e+09
220000000.0
raw_data [ '2017-10-01 15:30:01+00:00' : '2017-10-01 15:30:03+00:00' ]
LHC.BCTDC.A6R4.B1:BEAM_INTENSITY
LHC.BCTDC.A6R4.B2:BEAM_INTENSITY
2017-10-01 15:30:01+00:00
1.810000e+09
310000000.0
2017-10-01 15:30:02+00:00
1.690000e+09
-310000000.0
2017-10-01 15:30:03+00:00
1.990000e+09
-720000000.0
By default the index timezone is UTC but, even if not encouraged, you can chance the index time zone.
raw_data . index = raw_data . index . tz_convert ( 'CET' ) # CERN local time
raw_data . head ()
LHC.BCTDC.A6R4.B1:BEAM_INTENSITY
LHC.BCTDC.A6R4.B2:BEAM_INTENSITY
2017-10-01 17:30:00+02:00
1.470000e+09
270000000.0
2017-10-01 17:30:01+02:00
1.810000e+09
310000000.0
2017-10-01 17:30:02+02:00
1.690000e+09
-310000000.0
2017-10-01 17:30:03+02:00
1.990000e+09
-720000000.0
2017-10-01 17:30:04+02:00
1.270000e+09
220000000.0
You can split the query in different time windows
raw_data = importData . cals2pd ( variables , startTime , endTime , split = 4 , verbose = True )
Time window: 1
t1 is UTC localized: 2017-10-01 15:30:00+00:00
t2 is UTC localized: 2017-10-01 15:30:15+00:00
No fundamental filter.
Elaborating variable: LHC.BCTDC.A6R4.B1:BEAM_INTENSITY
Elaborating variable: LHC.BCTDC.A6R4.B2:BEAM_INTENSITY
Time window: 2
t1 is UTC localized: 2017-10-01 15:30:15+00:00
t2 is UTC localized: 2017-10-01 15:30:30+00:00
No fundamental filter.
Elaborating variable: LHC.BCTDC.A6R4.B1:BEAM_INTENSITY
Elaborating variable: LHC.BCTDC.A6R4.B2:BEAM_INTENSITY
Time window: 3
t1 is UTC localized: 2017-10-01 15:30:30+00:00
t2 is UTC localized: 2017-10-01 15:30:45+00:00
No fundamental filter.
Elaborating variable: LHC.BCTDC.A6R4.B1:BEAM_INTENSITY
Elaborating variable: LHC.BCTDC.A6R4.B2:BEAM_INTENSITY
Time window: 4
t1 is UTC localized: 2017-10-01 15:30:45+00:00
t2 is UTC localized: 2017-10-01 15:31:00+00:00
No fundamental filter.
Elaborating variable: LHC.BCTDC.A6R4.B1:BEAM_INTENSITY
Elaborating variable: LHC.BCTDC.A6R4.B2:BEAM_INTENSITY
You can filter by fundamentals.
t1 = pd . Timestamp ( '2017-10-01 17:30' , tz = 'CET' )
t2 = pd . Timestamp ( '2017-10-01 17:31' , tz = 'CET' )
raw_data = importData . cals2pd ([ 'CPS.TGM:DEST' ], t1 , t2 , fundamental = '%TOF%' )
raw_data
CPS.TGM:DEST
2017-10-01 15:30:13.900000095+00:00
NTOF
2017-10-01 15:30:17.500000+00:00
NTOF
2017-10-01 15:30:25.900000095+00:00
NTOF
2017-10-01 15:30:27.100000143+00:00
NTOF
2017-10-01 15:30:30.700000048+00:00
NTOF
2017-10-01 15:30:35.500000+00:00
NTOF
2017-10-01 15:30:49.900000095+00:00
NTOF
2017-10-01 15:30:53.500000+00:00
NTOF
cycleStamp2pd function
This function allow to retrieve information from CALS with a list of cyclestamp. For each element in the list it makes a call to pytimber (so can be slow). It is very useful to make aritmetic with the cyclestamps (e.g., for the CERN Injectors Chain).
startTime = pd . Timestamp ( '2018-03-27 06:00' )
endTime = pd . Timestamp ( '2018-03-27 06:10' )
CPSDF = importData . cals2pd ([ 'CPS.LSA:CYCLE' ], startTime , endTime , fundamental = '%LHC25%' )
importData . cycleStamp2pd ([ 'PSB.LSA:CYCLE' ], CPSDF . index [ 1 :] - pd . offsets . Milli ( 635 )) . head ()
PSB.LSA:CYCLE
2018-03-27 06:00:36.065000057+00:00
LHC25_DB_A_PSB
2018-03-27 06:01:13.265000105+00:00
LHC25_DB_A_PSB
2018-03-27 06:01:46.865000010+00:00
LHC25_DB_A_PSB
2018-03-27 06:01:50.464999914+00:00
LHC25_DB_A_PSB
2018-03-27 06:02:24.065000057+00:00
LHC25_DB_A_PSB
LHCFillsByTime function
One can recover the LHC fills between two timestamps.
t1 = pd . Timestamp ( '2018-04-24 16:57:54.689000+00:00' )
t2 = pd . Timestamp ( '2018-05-24 16:57:54.689000+00:00' )
dfFILL = importData . LHCFillsByTime ( t1 , t2 )
dfFILL
mode
startTime
endTime
duration
6611
FILL
2018-04-24 16:57:54.688999891+00:00
2018-04-25 07:15:18.815999985+00:00
14:17:24.127000
6611
SETUP
2018-04-24 17:16:51.466000080+00:00
2018-04-24 18:56:27.641000032+00:00
01:39:36.174999
6611
INJPROT
2018-04-24 18:56:27.641999960+00:00
2018-04-24 19:13:20.552999973+00:00
00:16:52.911000
6611
INJPHYS
2018-04-24 19:13:20.553999901+00:00
2018-04-24 19:52:14.631000042+00:00
00:38:54.077000
6611
PRERAMP
2018-04-24 19:52:14.631999969+00:00
2018-04-24 19:54:30.025000095+00:00
00:02:15.393000
6611
RAMP
2018-04-24 19:54:30.026000023+00:00
2018-04-24 20:14:44.450000048+00:00
00:20:14.424000
6611
FLATTOP
2018-04-24 20:14:44.450999975+00:00
2018-04-24 20:17:40.210000038+00:00
00:02:55.759000
6611
SQUEEZE
2018-04-24 20:17:40.210999966+00:00
2018-04-24 20:28:38.168999910+00:00
00:10:57.957999
6611
ADJUST
2018-04-24 20:28:38.170000076+00:00
2018-04-24 20:35:27.440000057+00:00
00:06:49.269999
6611
STABLE
2018-04-24 20:35:27.440999985+00:00
2018-04-25 06:00:49.973999977+00:00
09:25:22.532999
6611
ADJUST
2018-04-25 06:00:49.974999905+00:00
2018-04-25 07:09:16.572999954+00:00
01:08:26.598000
6611
BEAMDUMP
2018-04-25 07:09:16.573999882+00:00
2018-04-25 07:10:50.769000053+00:00
00:01:34.195000
6611
RAMPDOWN
2018-04-25 07:10:50.769999981+00:00
2018-04-25 07:15:18.815999985+00:00
00:04:28.046000
6612
FILL
2018-04-25 07:15:18.816999912+00:00
2018-04-25 20:58:43.336999893+00:00
13:43:24.519999
6612
SETUP
2018-04-25 07:47:43.690000057+00:00
2018-04-25 08:24:19.234999895+00:00
00:36:35.544999
6612
INJPROT
2018-04-25 08:24:19.236000061+00:00
2018-04-25 08:55:44.729000092+00:00
00:31:25.493000
6612
SETUP
2018-04-25 08:55:44.730000019+00:00
2018-04-25 09:00:50.864000082+00:00
00:05:06.134000
6612
SETUP
2018-04-25 09:00:50.865000010+00:00
2018-04-25 09:06:37.197999954+00:00
00:05:46.332999
6612
INJPROT
2018-04-25 09:06:37.198999882+00:00
2018-04-25 10:17:40.709000111+00:00
01:11:03.510000
6612
INJPHYS
2018-04-25 10:17:40.710000038+00:00
2018-04-25 11:24:47.825999975+00:00
01:07:07.115999
6612
PRERAMP
2018-04-25 11:24:47.826999903+00:00
2018-04-25 11:29:46.378000021+00:00
00:04:58.551000
6612
RAMP
2018-04-25 11:29:46.378999949+00:00
2018-04-25 11:50:12.374000072+00:00
00:20:25.995000
6612
FLATTOP
2018-04-25 11:50:12.375000+00:00
2018-04-25 11:57:10.351999998+00:00
00:06:57.976999
6612
SQUEEZE
2018-04-25 11:57:10.352999926+00:00
2018-04-25 12:08:05.000999928+00:00
00:10:54.648000
6612
ADJUST
2018-04-25 12:08:05.002000093+00:00
2018-04-25 12:16:59.278000116+00:00
00:08:54.276000
6612
STABLE
2018-04-25 12:16:59.279000044+00:00
2018-04-25 20:53:16.812999964+00:00
08:36:17.533999
6612
BEAMDUMP
2018-04-25 20:53:16.813999891+00:00
2018-04-25 20:54:39.635999918+00:00
00:01:22.822000
6612
RAMPDOWN
2018-04-25 20:54:39.637000084+00:00
2018-04-25 20:58:43.336999893+00:00
00:04:03.699999
6613
FILL
2018-04-25 20:58:43.338000059+00:00
2018-04-25 23:57:48.444999933+00:00
02:59:05.106999
6613
SETUP
2018-04-25 22:00:32.696000099+00:00
2018-04-25 22:09:09.723000050+00:00
00:08:37.026999
...
...
...
...
...
6712
SQUEEZE
2018-05-22 16:23:08.066999911+00:00
2018-05-22 16:34:02.688999891+00:00
00:10:54.621999
6712
ADJUST
2018-05-22 16:34:02.690000057+00:00
2018-05-22 16:41:52.858999968+00:00
00:07:50.168999
6712
STABLE
2018-05-22 16:41:52.859999895+00:00
2018-05-23 03:07:48.640000105+00:00
10:25:55.780000
6712
BEAMDUMP
2018-05-23 03:07:48.641000032+00:00
2018-05-23 03:08:02.177000046+00:00
00:00:13.536000
6712
RAMPDOWN
2018-05-23 03:08:02.177999973+00:00
2018-05-23 03:12:09.263999939+00:00
00:04:07.085999
6713
FILL
2018-05-23 03:12:09.265000105+00:00
2018-05-23 04:27:56.147000074+00:00
01:15:46.881999
6713
SETUP
2018-05-23 03:27:44.332000017+00:00
2018-05-23 03:47:09.704999924+00:00
00:19:25.372999
6713
INJPROT
2018-05-23 03:47:09.706000090+00:00
2018-05-23 03:57:04.144000053+00:00
00:09:54.437999
6713
INJPHYS
2018-05-23 03:57:04.144999981+00:00
2018-05-23 04:27:56.147000074+00:00
00:30:52.002000
6714
FILL
2018-05-23 04:27:56.148000002+00:00
2018-05-23 19:52:48.005000114+00:00
15:24:51.857000
6714
INJPROT
2018-05-23 04:28:16.176000118+00:00
2018-05-23 04:31:35.838000059+00:00
00:03:19.661999
6714
INJPHYS
2018-05-23 04:31:35.838999987+00:00
2018-05-23 05:00:19.354000092+00:00
00:28:43.515000
6714
PRERAMP
2018-05-23 05:00:19.355000019+00:00
2018-05-23 05:03:33.286999941+00:00
00:03:13.931999
6714
RAMP
2018-05-23 05:03:33.288000107+00:00
2018-05-23 05:23:47.737999916+00:00
00:20:14.449999
6714
FLATTOP
2018-05-23 05:23:47.739000082+00:00
2018-05-23 05:27:13.437000036+00:00
00:03:25.697999
6714
SQUEEZE
2018-05-23 05:27:13.437999964+00:00
2018-05-23 05:38:08.301000118+00:00
00:10:54.863000
6714
ADJUST
2018-05-23 05:38:08.302000046+00:00
2018-05-23 05:49:04.497999907+00:00
00:10:56.195999
6714
STABLE
2018-05-23 05:49:04.499000072+00:00
2018-05-23 19:48:11.851999998+00:00
13:59:07.352999
6714
BEAMDUMP
2018-05-23 19:48:11.852999926+00:00
2018-05-23 19:48:40.487999916+00:00
00:00:28.634999
6714
RAMPDOWN
2018-05-23 19:48:40.489000082+00:00
2018-05-23 19:52:48.005000114+00:00
00:04:07.516000
6715
FILL
2018-05-23 19:52:48.006000042+00:00
2018-05-23 22:47:48.658999920+00:00
02:55:00.652999
6715
SETUP
2018-05-23 20:23:27.148999929+00:00
2018-05-23 21:28:07.967999935+00:00
01:04:40.819000
6715
INJPROT
2018-05-23 21:28:07.969000101+00:00
2018-05-23 22:20:01.910000086+00:00
00:51:53.940999
6715
INJPHYS
2018-05-23 22:20:01.911000013+00:00
2018-05-23 22:47:48.658999920+00:00
00:27:46.747999
6716
FILL
2018-05-23 22:47:48.660000086+00:00
2018-05-24 04:15:31.756000042+00:00
05:27:43.095999
6716
INJPROT
2018-05-23 22:48:01.979000092+00:00
2018-05-24 02:11:03.315000057+00:00
03:23:01.335999
6716
NOBEAM
2018-05-24 02:11:03.315999985+00:00
2018-05-24 04:15:29.599999905+00:00
02:04:26.283999
6716
CYCLING
2018-05-24 04:15:29.601000071+00:00
2018-05-24 04:15:31.756000042+00:00
00:00:02.154999
6717
FILL
2018-05-24 04:15:31.756999969+00:00
2018-05-24 18:49:18.076999903+00:00
14:33:46.319999
6717
NOBEAM
2018-05-24 05:37:40.792000055+00:00
2018-05-24 18:49:18.076999903+00:00
13:11:37.284999
925 rows × 4 columns
On can change the time zone (in case the user want to do it...)
def _CETconvertMe ( x ):
'''
Return the tz-aware datetime. In case of error returns x.
'''
try :
return x . tz_convert ( 'CET' )
except :
return x
dfFILL [ 'startTime' ] = dfFILL [ 'startTime' ] . apply ( _CETconvertMe )
dfFILL [ 'endTime' ] = dfFILL [ 'endTime' ] . apply ( _CETconvertMe )
dfFILL
mode
startTime
endTime
duration
6611
FILL
2018-04-24 18:57:54.688999891+02:00
2018-04-25 09:15:18.815999985+02:00
14:17:24.127000
6611
SETUP
2018-04-24 19:16:51.466000080+02:00
2018-04-24 20:56:27.641000032+02:00
01:39:36.174999
6611
INJPROT
2018-04-24 20:56:27.641999960+02:00
2018-04-24 21:13:20.552999973+02:00
00:16:52.911000
6611
INJPHYS
2018-04-24 21:13:20.553999901+02:00
2018-04-24 21:52:14.631000042+02:00
00:38:54.077000
6611
PRERAMP
2018-04-24 21:52:14.631999969+02:00
2018-04-24 21:54:30.025000095+02:00
00:02:15.393000
6611
RAMP
2018-04-24 21:54:30.026000023+02:00
2018-04-24 22:14:44.450000048+02:00
00:20:14.424000
6611
FLATTOP
2018-04-24 22:14:44.450999975+02:00
2018-04-24 22:17:40.210000038+02:00
00:02:55.759000
6611
SQUEEZE
2018-04-24 22:17:40.210999966+02:00
2018-04-24 22:28:38.168999910+02:00
00:10:57.957999
6611
ADJUST
2018-04-24 22:28:38.170000076+02:00
2018-04-24 22:35:27.440000057+02:00
00:06:49.269999
6611
STABLE
2018-04-24 22:35:27.440999985+02:00
2018-04-25 08:00:49.973999977+02:00
09:25:22.532999
6611
ADJUST
2018-04-25 08:00:49.974999905+02:00
2018-04-25 09:09:16.572999954+02:00
01:08:26.598000
6611
BEAMDUMP
2018-04-25 09:09:16.573999882+02:00
2018-04-25 09:10:50.769000053+02:00
00:01:34.195000
6611
RAMPDOWN
2018-04-25 09:10:50.769999981+02:00
2018-04-25 09:15:18.815999985+02:00
00:04:28.046000
6612
FILL
2018-04-25 09:15:18.816999912+02:00
2018-04-25 22:58:43.336999893+02:00
13:43:24.519999
6612
SETUP
2018-04-25 09:47:43.690000057+02:00
2018-04-25 10:24:19.234999895+02:00
00:36:35.544999
6612
INJPROT
2018-04-25 10:24:19.236000061+02:00
2018-04-25 10:55:44.729000092+02:00
00:31:25.493000
6612
SETUP
2018-04-25 10:55:44.730000019+02:00
2018-04-25 11:00:50.864000082+02:00
00:05:06.134000
6612
SETUP
2018-04-25 11:00:50.865000010+02:00
2018-04-25 11:06:37.197999954+02:00
00:05:46.332999
6612
INJPROT
2018-04-25 11:06:37.198999882+02:00
2018-04-25 12:17:40.709000111+02:00
01:11:03.510000
6612
INJPHYS
2018-04-25 12:17:40.710000038+02:00
2018-04-25 13:24:47.825999975+02:00
01:07:07.115999
6612
PRERAMP
2018-04-25 13:24:47.826999903+02:00
2018-04-25 13:29:46.378000021+02:00
00:04:58.551000
6612
RAMP
2018-04-25 13:29:46.378999949+02:00
2018-04-25 13:50:12.374000072+02:00
00:20:25.995000
6612
FLATTOP
2018-04-25 13:50:12.375000+02:00
2018-04-25 13:57:10.351999998+02:00
00:06:57.976999
6612
SQUEEZE
2018-04-25 13:57:10.352999926+02:00
2018-04-25 14:08:05.000999928+02:00
00:10:54.648000
6612
ADJUST
2018-04-25 14:08:05.002000093+02:00
2018-04-25 14:16:59.278000116+02:00
00:08:54.276000
6612
STABLE
2018-04-25 14:16:59.279000044+02:00
2018-04-25 22:53:16.812999964+02:00
08:36:17.533999
6612
BEAMDUMP
2018-04-25 22:53:16.813999891+02:00
2018-04-25 22:54:39.635999918+02:00
00:01:22.822000
6612
RAMPDOWN
2018-04-25 22:54:39.637000084+02:00
2018-04-25 22:58:43.336999893+02:00
00:04:03.699999
6613
FILL
2018-04-25 22:58:43.338000059+02:00
2018-04-26 01:57:48.444999933+02:00
02:59:05.106999
6613
SETUP
2018-04-26 00:00:32.696000099+02:00
2018-04-26 00:09:09.723000050+02:00
00:08:37.026999
...
...
...
...
...
6712
SQUEEZE
2018-05-22 18:23:08.066999911+02:00
2018-05-22 18:34:02.688999891+02:00
00:10:54.621999
6712
ADJUST
2018-05-22 18:34:02.690000057+02:00
2018-05-22 18:41:52.858999968+02:00
00:07:50.168999
6712
STABLE
2018-05-22 18:41:52.859999895+02:00
2018-05-23 05:07:48.640000105+02:00
10:25:55.780000
6712
BEAMDUMP
2018-05-23 05:07:48.641000032+02:00
2018-05-23 05:08:02.177000046+02:00
00:00:13.536000
6712
RAMPDOWN
2018-05-23 05:08:02.177999973+02:00
2018-05-23 05:12:09.263999939+02:00
00:04:07.085999
6713
FILL
2018-05-23 05:12:09.265000105+02:00
2018-05-23 06:27:56.147000074+02:00
01:15:46.881999
6713
SETUP
2018-05-23 05:27:44.332000017+02:00
2018-05-23 05:47:09.704999924+02:00
00:19:25.372999
6713
INJPROT
2018-05-23 05:47:09.706000090+02:00
2018-05-23 05:57:04.144000053+02:00
00:09:54.437999
6713
INJPHYS
2018-05-23 05:57:04.144999981+02:00
2018-05-23 06:27:56.147000074+02:00
00:30:52.002000
6714
FILL
2018-05-23 06:27:56.148000002+02:00
2018-05-23 21:52:48.005000114+02:00
15:24:51.857000
6714
INJPROT
2018-05-23 06:28:16.176000118+02:00
2018-05-23 06:31:35.838000059+02:00
00:03:19.661999
6714
INJPHYS
2018-05-23 06:31:35.838999987+02:00
2018-05-23 07:00:19.354000092+02:00
00:28:43.515000
6714
PRERAMP
2018-05-23 07:00:19.355000019+02:00
2018-05-23 07:03:33.286999941+02:00
00:03:13.931999
6714
RAMP
2018-05-23 07:03:33.288000107+02:00
2018-05-23 07:23:47.737999916+02:00
00:20:14.449999
6714
FLATTOP
2018-05-23 07:23:47.739000082+02:00
2018-05-23 07:27:13.437000036+02:00
00:03:25.697999
6714
SQUEEZE
2018-05-23 07:27:13.437999964+02:00
2018-05-23 07:38:08.301000118+02:00
00:10:54.863000
6714
ADJUST
2018-05-23 07:38:08.302000046+02:00
2018-05-23 07:49:04.497999907+02:00
00:10:56.195999
6714
STABLE
2018-05-23 07:49:04.499000072+02:00
2018-05-23 21:48:11.851999998+02:00
13:59:07.352999
6714
BEAMDUMP
2018-05-23 21:48:11.852999926+02:00
2018-05-23 21:48:40.487999916+02:00
00:00:28.634999
6714
RAMPDOWN
2018-05-23 21:48:40.489000082+02:00
2018-05-23 21:52:48.005000114+02:00
00:04:07.516000
6715
FILL
2018-05-23 21:52:48.006000042+02:00
2018-05-24 00:47:48.658999920+02:00
02:55:00.652999
6715
SETUP
2018-05-23 22:23:27.148999929+02:00
2018-05-23 23:28:07.967999935+02:00
01:04:40.819000
6715
INJPROT
2018-05-23 23:28:07.969000101+02:00
2018-05-24 00:20:01.910000086+02:00
00:51:53.940999
6715
INJPHYS
2018-05-24 00:20:01.911000013+02:00
2018-05-24 00:47:48.658999920+02:00
00:27:46.747999
6716
FILL
2018-05-24 00:47:48.660000086+02:00
2018-05-24 06:15:31.756000042+02:00
05:27:43.095999
6716
INJPROT
2018-05-24 00:48:01.979000092+02:00
2018-05-24 04:11:03.315000057+02:00
03:23:01.335999
6716
NOBEAM
2018-05-24 04:11:03.315999985+02:00
2018-05-24 06:15:29.599999905+02:00
02:04:26.283999
6716
CYCLING
2018-05-24 06:15:29.601000071+02:00
2018-05-24 06:15:31.756000042+02:00
00:00:02.154999
6717
FILL
2018-05-24 06:15:31.756999969+02:00
2018-05-24 20:49:18.076999903+02:00
14:33:46.319999
6717
NOBEAM
2018-05-24 07:37:40.792000055+02:00
2018-05-24 20:49:18.076999903+02:00
13:11:37.284999
925 rows × 4 columns
LHCFillsByNumber function
One can select the LHC fill by fill number.
dfFILL = importData . LHCFillsByNumber ( 6611 )
dfFILL
mode
startTime
endTime
duration
6611
FILL
2018-04-24 16:57:54.688999891+00:00
2018-04-25 07:15:18.815999985+00:00
14:17:24.127000
6611
SETUP
2018-04-24 17:16:51.466000080+00:00
2018-04-24 18:56:27.641000032+00:00
01:39:36.174999
6611
INJPROT
2018-04-24 18:56:27.641999960+00:00
2018-04-24 19:13:20.552999973+00:00
00:16:52.911000
6611
INJPHYS
2018-04-24 19:13:20.553999901+00:00
2018-04-24 19:52:14.631000042+00:00
00:38:54.077000
6611
PRERAMP
2018-04-24 19:52:14.631999969+00:00
2018-04-24 19:54:30.025000095+00:00
00:02:15.393000
6611
RAMP
2018-04-24 19:54:30.026000023+00:00
2018-04-24 20:14:44.450000048+00:00
00:20:14.424000
6611
FLATTOP
2018-04-24 20:14:44.450999975+00:00
2018-04-24 20:17:40.210000038+00:00
00:02:55.759000
6611
SQUEEZE
2018-04-24 20:17:40.210999966+00:00
2018-04-24 20:28:38.168999910+00:00
00:10:57.957999
6611
ADJUST
2018-04-24 20:28:38.170000076+00:00
2018-04-24 20:35:27.440000057+00:00
00:06:49.269999
6611
STABLE
2018-04-24 20:35:27.440999985+00:00
2018-04-25 06:00:49.973999977+00:00
09:25:22.532999
6611
ADJUST
2018-04-25 06:00:49.974999905+00:00
2018-04-25 07:09:16.572999954+00:00
01:08:26.598000
6611
BEAMDUMP
2018-04-25 07:09:16.573999882+00:00
2018-04-25 07:10:50.769000053+00:00
00:01:34.195000
6611
RAMPDOWN
2018-04-25 07:10:50.769999981+00:00
2018-04-25 07:15:18.815999985+00:00
00:04:28.046000
dfFILL = importData . LHCFillsByNumber ([ 6611 , 6666 ])
dfFILL
mode
startTime
endTime
duration
6611
FILL
2018-04-24 16:57:54.688999891+00:00
2018-04-25 07:15:18.815999985+00:00
14:17:24.127000
6611
SETUP
2018-04-24 17:16:51.466000080+00:00
2018-04-24 18:56:27.641000032+00:00
01:39:36.174999
6611
INJPROT
2018-04-24 18:56:27.641999960+00:00
2018-04-24 19:13:20.552999973+00:00
00:16:52.911000
6611
INJPHYS
2018-04-24 19:13:20.553999901+00:00
2018-04-24 19:52:14.631000042+00:00
00:38:54.077000
6611
PRERAMP
2018-04-24 19:52:14.631999969+00:00
2018-04-24 19:54:30.025000095+00:00
00:02:15.393000
6611
RAMP
2018-04-24 19:54:30.026000023+00:00
2018-04-24 20:14:44.450000048+00:00
00:20:14.424000
6611
FLATTOP
2018-04-24 20:14:44.450999975+00:00
2018-04-24 20:17:40.210000038+00:00
00:02:55.759000
6611
SQUEEZE
2018-04-24 20:17:40.210999966+00:00
2018-04-24 20:28:38.168999910+00:00
00:10:57.957999
6611
ADJUST
2018-04-24 20:28:38.170000076+00:00
2018-04-24 20:35:27.440000057+00:00
00:06:49.269999
6611
STABLE
2018-04-24 20:35:27.440999985+00:00
2018-04-25 06:00:49.973999977+00:00
09:25:22.532999
6611
ADJUST
2018-04-25 06:00:49.974999905+00:00
2018-04-25 07:09:16.572999954+00:00
01:08:26.598000
6611
BEAMDUMP
2018-04-25 07:09:16.573999882+00:00
2018-04-25 07:10:50.769000053+00:00
00:01:34.195000
6611
RAMPDOWN
2018-04-25 07:10:50.769999981+00:00
2018-04-25 07:15:18.815999985+00:00
00:04:28.046000
6666
FILL
2018-05-10 19:35:38.635999918+00:00
2018-05-11 10:29:36.956000090+00:00
14:53:58.320000
6666
INJPROT
2018-05-10 19:35:51.628000021+00:00
2018-05-10 19:39:39.914999962+00:00
00:03:48.286999
6666
INJPHYS
2018-05-10 19:39:39.915999889+00:00
2018-05-10 20:25:05.993000031+00:00
00:45:26.077000
6666
PRERAMP
2018-05-10 20:25:05.993999958+00:00
2018-05-10 20:33:55.484999895+00:00
00:08:49.490999
6666
RAMP
2018-05-10 20:33:55.486000061+00:00
2018-05-10 20:54:09.895999908+00:00
00:20:14.409999
6666
FLATTOP
2018-05-10 20:54:09.897000074+00:00
2018-05-10 20:56:43.071000099+00:00
00:02:33.174000
6666
SQUEEZE
2018-05-10 20:56:43.072000027+00:00
2018-05-10 21:07:37.651000023+00:00
00:10:54.578999
6666
ADJUST
2018-05-10 21:07:37.651999950+00:00
2018-05-10 21:13:16.588999987+00:00
00:05:38.937000
6666
STABLE
2018-05-10 21:13:16.589999914+00:00
2018-05-11 09:56:28.926000118+00:00
12:43:12.336000
6666
ADJUST
2018-05-11 09:56:28.927000046+00:00
2018-05-11 10:22:51.443000078+00:00
00:26:22.516000
6666
BEAMDUMP
2018-05-11 10:22:51.444000006+00:00
2018-05-11 10:24:59.108000040+00:00
00:02:07.664000
6666
BEAMDUMP
2018-05-11 10:24:59.108999968+00:00
2018-05-11 10:25:15.894000053+00:00
00:00:16.785000
6666
RAMPDOWN
2018-05-11 10:25:15.894999981+00:00
2018-05-11 10:29:36.956000090+00:00
00:04:21.061000
LHCInstant
To retrieve the status of the LHC (fill and beammode at a specific time).
t1 = pd . Timestamp ( '2018-05-22 02:10:15' , tz = 'CET' )
importData . LHCInstant ( t1 )
mode
startTime
endTime
duration
6711
INJPHYS
2018-05-21 23:23:11.940000057+00:00
2018-05-22 00:27:06.229000092+00:00
01:03:54.289000
LHCCals2pd
To collect datas for specific LHC fills or beam modes.
importData . LHCCals2pd ([ '%.RQX.%1:I_MEAS' ],[ 6278 ],[ 'RAMP' , 'FLATTOP' ], fill_column = True , beamMode_column = True )
RPHFC.UL14.RQX.L1:I_MEAS
RPHFC.UL16.RQX.R1:I_MEAS
fill
mode
2017-10-06 21:25:06.059999943+00:00
NaN
407.31
6278
RAMP
2017-10-06 21:25:06.079999924+00:00
407.37
NaN
6278
RAMP
2017-10-06 21:25:06.559999943+00:00
NaN
407.38
6278
RAMP
2017-10-06 21:25:06.579999924+00:00
407.45
NaN
6278
RAMP
2017-10-06 21:25:10.059999943+00:00
NaN
408.06
6278
RAMP
2017-10-06 21:25:10.079999924+00:00
408.12
NaN
6278
RAMP
2017-10-06 21:25:10.559999943+00:00
NaN
408.18
6278
RAMP
2017-10-06 21:25:10.579999924+00:00
408.24
NaN
6278
RAMP
2017-10-06 21:25:13.059999943+00:00
NaN
408.82
6278
RAMP
2017-10-06 21:25:13.079999924+00:00
408.89
NaN
6278
RAMP
2017-10-06 21:25:13.559999943+00:00
NaN
408.96
6278
RAMP
2017-10-06 21:25:13.579999924+00:00
409.02
NaN
6278
RAMP
2017-10-06 21:25:15.559999943+00:00
NaN
409.57
6278
RAMP
2017-10-06 21:25:15.579999924+00:00
409.64
NaN
6278
RAMP
2017-10-06 21:25:16.059999943+00:00
NaN
409.74
6278
RAMP
2017-10-06 21:25:16.079999924+00:00
409.81
NaN
6278
RAMP
2017-10-06 21:25:17.559999943+00:00
NaN
410.26
6278
RAMP
2017-10-06 21:25:17.579999924+00:00
410.33
NaN
6278
RAMP
2017-10-06 21:25:18.059999943+00:00
NaN
410.44
6278
RAMP
2017-10-06 21:25:18.079999924+00:00
410.51
NaN
6278
RAMP
2017-10-06 21:25:19.559999943+00:00
NaN
411.02
6278
RAMP
2017-10-06 21:25:19.579999924+00:00
411.09
NaN
6278
RAMP
2017-10-06 21:25:20.059999943+00:00
NaN
411.22
6278
RAMP
2017-10-06 21:25:20.079999924+00:00
411.29
NaN
6278
RAMP
2017-10-06 21:25:21.559999943+00:00
NaN
411.84
6278
RAMP
2017-10-06 21:25:21.579999924+00:00
411.91
NaN
6278
RAMP
2017-10-06 21:25:22.059999943+00:00
NaN
412.07
6278
RAMP
2017-10-06 21:25:22.079999924+00:00
412.14
NaN
6278
RAMP
2017-10-06 21:25:23.559999943+00:00
NaN
412.75
6278
RAMP
2017-10-06 21:25:23.579999924+00:00
412.82
NaN
6278
RAMP
...
...
...
...
...
2017-10-06 21:44:58.559999943+00:00
NaN
6254.74
6278
RAMP
2017-10-06 21:44:58.579999924+00:00
6256.49
NaN
6278
RAMP
2017-10-06 21:44:59.059999943+00:00
NaN
6255.41
6278
RAMP
2017-10-06 21:44:59.079999924+00:00
6257.15
NaN
6278
RAMP
2017-10-06 21:44:59.559999943+00:00
NaN
6256.01
6278
RAMP
2017-10-06 21:44:59.579999924+00:00
6257.75
NaN
6278
RAMP
2017-10-06 21:45:00.059999943+00:00
NaN
6256.54
6278
RAMP
2017-10-06 21:45:00.079999924+00:00
6258.28
NaN
6278
RAMP
2017-10-06 21:45:00.559999943+00:00
NaN
6257.00
6278
RAMP
2017-10-06 21:45:00.579999924+00:00
6258.74
NaN
6278
RAMP
2017-10-06 21:45:01.059999943+00:00
NaN
6257.40
6278
RAMP
2017-10-06 21:45:01.079999924+00:00
6259.14
NaN
6278
RAMP
2017-10-06 21:45:01.559999943+00:00
NaN
6257.73
6278
RAMP
2017-10-06 21:45:01.579999924+00:00
6259.46
NaN
6278
RAMP
2017-10-06 21:46:01.059999943+00:00
NaN
6258.37
6278
FLATTOP
2017-10-06 21:46:01.079999924+00:00
6260.10
NaN
6278
FLATTOP
2017-10-06 21:46:01.559999943+00:00
NaN
6258.37
6278
FLATTOP
2017-10-06 21:46:01.579999924+00:00
6260.10
NaN
6278
FLATTOP
2017-10-06 21:47:01.059999943+00:00
NaN
6258.37
6278
FLATTOP
2017-10-06 21:47:01.079999924+00:00
6260.10
NaN
6278
FLATTOP
2017-10-06 21:47:01.559999943+00:00
NaN
6258.37
6278
FLATTOP
2017-10-06 21:47:01.579999924+00:00
6260.10
NaN
6278
FLATTOP
2017-10-06 21:48:01.059999943+00:00
NaN
6258.37
6278
FLATTOP
2017-10-06 21:48:01.079999924+00:00
6260.10
NaN
6278
FLATTOP
2017-10-06 21:48:01.559999943+00:00
NaN
6258.37
6278
FLATTOP
2017-10-06 21:48:01.579999924+00:00
6260.10
NaN
6278
FLATTOP
2017-10-06 21:49:01.059999943+00:00
NaN
6258.37
6278
FLATTOP
2017-10-06 21:49:01.079999924+00:00
6260.10
NaN
6278
FLATTOP
2017-10-06 21:49:01.559999943+00:00
NaN
6258.37
6278
FLATTOP
2017-10-06 21:49:01.579999924+00:00
6260.10
NaN
6278
FLATTOP
4742 rows × 4 columns
massiFile2pd function
One can select load massi files in pandas Dataframes.
ATLAS = importData . massiFile2pd ( '/eos/user/s/sterbini/MD_ANALYSIS/2017/LHC/MD2201/ATLAS_6195.tgz' )
FILL
Stable Beam Flag
Experiment
Bunch
Luminosity [Hz/ub]
P2P luminosity error [Hz/ub]
Specific luminosity [Hz/ub]
P2P specific luminosity [Hz/ub]
2017-09-13 18:00:43+00:00
6195
0
ATLAS
1490
0.000047
0.0
0.0
0.0
2017-09-13 18:01:43+00:00
6195
0
ATLAS
1490
0.000047
0.0
0.0
0.0
2017-09-13 18:21:26+00:00
6195
0
ATLAS
1490
0.000017
0.0
0.0
0.0
calsCSV2pd function
This allows to convert in pandas dataframes the csv data obtained from CALS.
myFile = '/eos/project/l/lhc-lumimod/LuminosityFollowUp/2017/rawdata/fill_bunchbybunch_data_csvs/bunchbybunch_lumi_data_fill_5848.csv'
aux = importData . calsCSV2pd ( myFile )
aux . head ()
ATLAS:BUNCH_LUMI_INST
CMS:BUNCH_LUMI_INST
2017-06-19 15:42:47.745000+00:00
[0.045945592, 0.045026667, 0.046864517, 0.0450...
NaN
2017-06-19 15:44:48.104000+00:00
[0.039506007, 0.04899983, 0.041649766, 0.04011...
NaN
2017-06-19 15:45:48.287000+00:00
[0.044709116, 0.04869012, 0.0523649, 0.0431779...
NaN
2017-06-19 15:47:48.234000+00:00
[0.03982115, 0.039514832, 0.042884354, 0.04870...
NaN
2017-06-19 15:48:48.298000+00:00
[0.05175795, 0.04410135, 0.049307834, 0.049614...
NaN
# to retrieve directly from CALS. This is for checks and in general is slower
aux = importData . cals2pd ( aux . columns , t1 = aux . index [ 0 ], t2 = aux . index [ 2 ])
aux
ATLAS:BUNCH_LUMI_INST
CMS:BUNCH_LUMI_INST
2017-06-19 15:42:47.745000124+00:00
[0.045945592, 0.045026667, 0.046864517, 0.0450...
NaN
2017-06-19 15:44:48.104000092+00:00
[0.039506007, 0.04899983, 0.041649766, 0.04011...
NaN
2017-06-19 15:45:48.286999941+00:00
[0.044709116, 0.04869012, 0.0523649, 0.0431779...
NaN
myFile = '/eos/project/l/lhc-lumimod/LuminosityFollowUp/2017/rawdata/fill_basic_data_csvs/basic_data_fill_5706.csv'
aux = importData . calsCSV2pd ( myFile )
aux . head ()
LHC.BCTDC.A6R4.B1:BEAM_INTENSITY
LHC.BCTDC.A6R4.B2:BEAM_INTENSITY
LHC.BSRA.US45.B1:ABORT_GAP_ENERGY
LHC.BSRA.US45.B2:ABORT_GAP_ENERGY
2017-05-25 13:01:13+00:00
2.410000e+09
590000000.0
NaN
NaN
2017-05-25 13:01:13.736000+00:00
NaN
NaN
450.0
450.0
2017-05-25 13:01:14+00:00
8.600000e+08
390000000.0
NaN
NaN
2017-05-25 13:01:14.736000+00:00
NaN
NaN
450.0
450.0
2017-05-25 13:01:15+00:00
1.300000e+09
-150000000.0
NaN
NaN
mat2dict function
Import a mat file in a mat_struct (very similar to a dict) that can be easily browsed.
aux = importData . mat2dict ( '/eos/user/s/sterbini/MD_ANALYSIS/2016/MD1780_80b/2016.10.26.22.23.42.135.mat' )
aux . CPS_BLM
<scipy.io.matlab.mio5_params.mat_struct at 0x7f84505ba128>
mat2pd function
aux = importData . mat2pd ([ 'CPS_BLM.Acquisition.value.lastLosses' ],[ '/eos/user/s/sterbini/MD_ANALYSIS/2016/MD1780_80b/2016.10.26.22.23.42.135.mat' ,
'/eos/user/s/sterbini/MD_ANALYSIS/2016/MD1780_80b/2016.10.26.22.23.06.147.mat' ])
aux
CPS_BLM.Acquisition.value.lastLosses
matlabFilePath
2016-10-26 20:22:27.100000+00:00
[53, 30, 95, 29, 12, 18, 3, 3, 12, 4, 2, 6, 0,...
/eos/user/s/sterbini/MD_ANALYSIS/2016/MD1780_8...
2016-10-26 20:23:03.100000+00:00
[58, 32, 93, 29, 12, 18, 3, 3, 13, 4, 2, 6, 0,...
/eos/user/s/sterbini/MD_ANALYSIS/2016/MD1780_8...
tfs2pd function
With this funtion we can import the the MADX TFS file in a pandas Dataframe.
aux = importData . tfs2pd ( '/eos/user/s/sterbini/MD_ANALYSIS/2018/LHC MD Optics/collisionAt25cm_180urad/lhcb1_thick.survey' )
display ( aux . transpose ())
aux . iloc [ 0 ][ 'TABLE' ] . head ()
The column SLOT_ID is empty.
The column ASSEMBLY_ID is empty.
/eos/user/s/sterbini/MD_ANALYSIS/2018/LHC MD Optics/collisionAt25cm_180urad/lhcb1_thick.survey
DATE
22/03/18
ORIGIN
5.03.07
TIME
22.30.11
TITLE
no-title
TYPE
SURVEY
TABLE
NAME KEYWORD ...
NAME
KEYWORD
S
L
ANGLE
X
Y
Z
THETA
PHI
PSI
GLOBALTILT
MECH_SEP
V_POS
0.000
IP1
MARKER
0.000
0.000
0.0
0.0
0.0
0.000
0.0
0.0
0.0
0.0
0.0
0.0
3.000
MBAS2.1R1
SOLENOID
3.000
3.000
0.0
0.0
0.0
3.000
0.0
0.0
0.0
0.0
0.0
0.0
19.050
DRIFT_0
DRIFT
19.050
16.050
0.0
0.0
0.0
19.050
0.0
0.0
0.0
0.0
0.0
0.0
20.850
TAS.1R1
RCOLLIMATOR
20.850
1.800
0.0
0.0
0.0
20.850
0.0
0.0
0.0
0.0
0.0
0.0
21.564
DRIFT_1
DRIFT
21.564
0.714
0.0
0.0
0.0
21.564
0.0
0.0
0.0
0.0
0.0
0.0
aux = importData . tfs2pd ( '/eos/user/s/sterbini/MD_ANALYSIS/2018/LHC MD Optics/collisionAt25cm_180urad/lhcb1_thick.twiss' )
aux . iloc [ 0 ][ 'TABLE' ] . head ()
NAME
PARENT
KEYWORD
S
L
X
Y
PX
PY
BETX
BETY
ALFX
ALFY
MUX
MUY
DX
DY
DPX
DPY
0.000
IP1
OMK
MARKER
0.000
0.000
-4.874885e-11
-1.062519e-12
-4.666460e-11
0.00018
0.250001
0.250000
-0.000002
1.233730e-07
0.000000
0.000000
0.026209
0.005558
0.01455
0.039107
3.000
MBAS2.1R1
MBAS2
SOLENOID
3.000
3.000
-1.887427e-10
5.400004e-04
-4.666460e-11
0.00018
36.249874
36.249965
-11.999956
-1.199999e+01
0.236767
0.236768
0.069860
0.122338
0.01455
0.039107
19.050
DRIFT_0
DRIFT
DRIFT
19.050
16.050
-9.377096e-10
3.429003e-03
-4.666460e-11
0.00018
1451.854515
1451.858614
-76.199710
-7.619993e+01
0.247911
0.247911
0.303393
0.747112
0.01455
0.039107
20.850
TAS.1R1
TAS
RCOLLIMATOR
20.850
1.800
-1.021706e-09
3.753003e-03
-4.666460e-11
0.00018
1739.133423
1739.138340
-83.399683
-8.339992e+01
0.248091
0.248092
0.329583
0.817180
0.01455
0.039107
21.564
DRIFT_1
DRIFT
DRIFT
21.564
0.714
-1.055024e-09
3.881523e-03
-4.666460e-11
0.00018
1860.267346
1860.272608
-86.255672
-8.625592e+01
0.248155
0.248155
0.339972
0.844974
0.01455
0.039107
aux = importData . tfs2pd ([ '/eos/user/s/sterbini/MD_ANALYSIS/2018/LHC MD Optics/collisionAt25cm_180urad/lhcb1_thick.survey' , '/eos/user/s/sterbini/MD_ANALYSIS/2018/LHC MD Optics/collisionAt25cm_180urad/lhcb1_thick.twiss' ])
The column SLOT_ID is empty.
The column ASSEMBLY_ID is empty.
aux . iloc [ 0 ][ 'TABLE' ] . index
Float64Index([ 0.0, 3.0, 19.05, 20.85, 21.564,
21.564, 21.564, 21.62, 21.62, 21.724,
...
26637.2482, 26637.2482, 26637.3042, 26637.3042, 26637.3042,
26638.0042, 26639.8042, 26655.8832, 26658.8832, 26658.8832],
dtype='float64', length=13283)
From dotdict to a simple pandas Dataframe.
We can transform a dotdict in a pandas Dataframe.
from cl2pd import dotdict
a = dotdict . dotdict ()
np = importData . np
for i in { 'B1' }:
a [ i ] = dotdict . dotdict ()
for j in range ( 10 ):
a [ i ][ 'b' + str ( j )] = np . random . randn ( 10 )
for i in { 'B2' }:
a [ i ] = dotdict . dotdict ()
for j in range ( 20 ):
a [ i ][ 'b' + str ( j )] = np . random . randn ( 10 )
B1
B2
b0
[0.6467321347905923, -1.0370676505843401, -1.5...
[1.5270813764876534, 1.2100639941492304, -1.11...
b1
[0.32969179854420916, 0.7858822252604646, -0.3...
[1.3064310857233223, -0.3359487143927113, 0.66...
b10
NaN
[-0.6957631261643009, 0.32370222762197626, 2.8...
b11
NaN
[-0.779053908334011, 1.506635143971471, -0.233...
b12
NaN
[2.0691097429781133, -1.528249449203943, 0.477...
b13
NaN
[-0.3705018424449959, -1.1877717848803848, -1....
b14
NaN
[-0.9949066537823384, -0.35825900064144167, -0...
b15
NaN
[0.21650090161488317, -0.5518600501885274, 0.5...
b16
NaN
[-0.6802440968350015, 0.7573210133031285, 0.53...
b17
NaN
[-2.2167781994862197, 0.6851372659010567, -0.0...
b18
NaN
[-1.7360108910276053, 0.3803327796866624, -1.6...
b19
NaN
[0.9476187485046135, 0.640806434726803, 0.5297...
b2
[-1.7182497708209206, 0.2865187814502491, -0.7...
[0.7847226477419496, -0.3710598283057992, -0.0...
b3
[-1.469273880140067, 0.4212359180496405, 1.037...
[0.05188168046413233, 1.4355051492894526, 2.11...
b4
[0.007974681615498608, -0.2610696151209144, 0....
[0.9520874475610571, -1.2910502451899022, -1.7...
b5
[0.49349416323479206, 0.8088796123324145, 0.19...
[-0.44431802009622257, 0.4787313869654191, 0.1...
b6
[0.5405258900294131, -0.5084350523854257, 0.14...
[-0.7898382541838375, 0.11480202852339073, -0....
b7
[-0.11305147458494907, 0.03314302262877794, -0...
[0.896925953242836, -0.14222940386632202, 0.18...
b8
[0.9405878186855667, 1.1799075215016352, 1.087...
[1.3164852160372085, -1.1817663457608947, 0.03...
b9
[-0.11927058554763645, -0.0652152524769821, 0....
[-0.4518129428607217, -0.16586580625802247, -0...
Concat check
variables = [ 'LHC.BCTDC.A6R4.B1:BEAM_INTENSITY' ]
startTime = pd . Timestamp ( '2017-10-01 17:30' , tz = 'CET' )
endTime = pd . Timestamp ( '2017-10-01 17:31' , tz = 'CET' )
raw_data1 = importData . cals2pd ( variables , startTime , endTime )
variables = [ 'LHC.BCTDC.A6R4.B2:BEAM_INTENSITY' ]
raw_data2 = importData . cals2pd ( variables , startTime , endTime )
pd . merge ( raw_data1 , raw_data2 , left_index = True , right_index = True ) . head ()
LHC.BCTDC.A6R4.B1:BEAM_INTENSITY
LHC.BCTDC.A6R4.B2:BEAM_INTENSITY
2017-10-01 15:30:00+00:00
1.470000e+09
270000000.0
2017-10-01 15:30:01+00:00
1.810000e+09
310000000.0
2017-10-01 15:30:02+00:00
1.690000e+09
-310000000.0
2017-10-01 15:30:03+00:00
1.990000e+09
-720000000.0
2017-10-01 15:30:04+00:00
1.270000e+09
220000000.0
# Interesting checks
# pd.Timestamp('2018-03-25 02:00', tz='CET') #this date does not exist, so the function returns an error.
# pd.Timestamp('2015-06-30 23:59:60', tz='UTC') #this date does exist (leap second) but it is not recognized.
In case of problem with pytimber
if 0 :
import cmmnbuild_dep_manager
mgr = cmmnbuild_dep_manager . Manager ()
mgr . install ( 'pytimber' )
mgr . resolve ()
from cl2pd import variablesDF
variablesDF . LHC
Variable
Tag
Type
On change
Description
0
LHC.BOFSU:OFC_ENERGY
Energy
NUMERIC
True
Beam energy
1
LHC.BCTFR.A6R4.B1:BEAM_INTENSITY
Intensity
NUMERIC
False
B1 intensity
2
LHC.BCTFR.A6R4.B2:BEAM_INTENSITY
Intensity
NUMERIC
False
B2 intensity
3
LHC.RUNCONFIG:IP1-XING-V-MURAD
Crossing angle
NUMERIC
True
Crossing angle IP1 V
4
LHC.RUNCONFIG:IP2-XING-V-MURAD
Crossing angle
NUMERIC
True
Crossing angle IP2 V
5
LHC.RUNCONFIG:IP5-XING-H-MURAD
Crossing angle
NUMERIC
True
Crossing angle IP5 H
6
LHC.RUNCONFIG:IP8-XING-H-MURAD
Crossing angle
NUMERIC
True
Crossing angle IP8 H
7
LHC.BQM.B1:BUNCH_LENGTH_MEAN
Bunch length
NUMERIC
False
Bunch Length B1
8
LHC.BQM.B2:BUNCH_LENGTH_MEAN
Bunch length
NUMERIC
False
Bunch Length B2
9
LHC.BQBBQ.CONTINUOUS.B1:TUNE_V
Tune
NUMERIC
False
Tune V B1 BBQ
10
LHC.BQBBQ.CONTINUOUS.B2:TUNE_V
Tune
NUMERIC
False
Tune V B2 BBQ
11
LHC.BQBBQ.CONTINUOUS_HS.B1:TUNE_V
Tune
NUMERIC
False
Most sensitive tune B1 V
12
LHC.BQBBQ.CONTINUOUS_HS.B2:TUNE_V
Tune
NUMERIC
False
Most sensitive tune B2 V
13
LHC.BQBBQ.CONTINUOUS.B1:TUNE_H
Tune
NUMERIC
False
Tune H B1 BBQ
14
LHC.BQBBQ.CONTINUOUS.B2:TUNE_H
Tune
NUMERIC
False
Tune H B2 BBQ
15
LHC.BQBBQ.CONTINUOUS_HS.B1:TUNE_H
Tune
NUMERIC
False
Most sensitive tune B1 H
16
LHC.BQBBQ.CONTINUOUS_HS.B2:TUNE_H
Tune
NUMERIC
False
Most sensitive tune B2 H
17
LHC.BQBBQ.CONTINUOUS.B1:FFT_DATA_H
Tune FFT
NUMERIC
False
Tune FFT H B1
18
LHC.BQBBQ.CONTINUOUS.B1:FFT_DATA_V
Tune FFT
NUMERIC
False
Tune FFT V B1
19
LHC.BQBBQ.CONTINUOUS.B2:FFT_DATA_H
Tune FFT
NUMERIC
False
Tune FFT H B2
20
LHC.BQBBQ.CONTINUOUS.B2:FFT_DATA_V
Tune FFT
NUMERIC
False
Tune FFT V B2
21
LHC.BCTFR.A6R4.B1:BUNCH_INTENSITY
Intensity
VECTOR NUMERIC
False
Main FBCT B1
22
LHC.BCTFR.A6R4.B2:BUNCH_INTENSITY
Intensity
VECTOR NUMERIC
False
Main FBCT B2
23
LHC.BCTFR.B6R4.B1:BUNCH_INTENSITY
Intensity
VECTOR NUMERIC
False
Spare FBCT B1
24
LHC.BCTFR.A6R4.B2:BUNCH_INTENSITY
Intensity
VECTOR NUMERIC
False
Spare FBCT B2
25
LHC.BCTFR.A6R4.B1:BUNCH_FILL_PATTERN
Filling Pattern
NUMERIC
False
Main FBCT B1
26
LHC.BCTFR.A6R4.B2:BUNCH_FILL_PATTERN
Filling Pattern
NUMERIC
False
Main FBCT B2
27
LHC.BCTFR.B6R4.B1:BUNCH_FILL_PATTERN
Filling Pattern
NUMERIC
False
Spare FBCT B1
28
LHC.BCTFR.A6R4.B2:BUNCH_FILL_PATTERN
Filling Pattern
NUMERIC
False
Spare FBCT B2
29
CMS:BUNCH_LUMI_INST
Luminosity
VECTOR NUMERIC
False
CMS bbb luminosity
...
...
...
...
...
...
124
ADTH.SR4.M1.B2:MDSPU_PHASE1
ADT Phase
NUMERIC
False
B2 Hozirontal ADT Phase 1 Mon1 (deg)
125
ADTH.SR4.M1.B2:MDSPU_PHASE2
ADT Phase
NUMERIC
False
B2 Hozirontal ADT Phase 2 Mon1 (deg)
126
ADTH.SR4.M1.B2:MDSPU_PHASE3
ADT Phase
NUMERIC
False
B2 Hozirontal ADT Phase 3 Mon1 (deg)
127
ADTH.SR4.M1.B2:MDSPU_PHASE4
ADT Phase
NUMERIC
False
B2 Hozirontal ADT Phase 4 Mon1 (deg)
128
ADTV.SR4.M1.B1:MDSPU_PHASE1
ADT Phase
NUMERIC
False
B2 Vertical ADT Phase 1 Mon1 (deg)
129
ADTV.SR4.M1.B1:MDSPU_PHASE2
ADT Phase
NUMERIC
False
B2 Vertical ADT Phase 2 Mon1 (deg)
130
ADTV.SR4.M1.B1:MDSPU_PHASE3
ADT Phase
NUMERIC
False
B2 Vertical ADT Phase 3 Mon1 (deg)
131
ADTV.SR4.M1.B1:MDSPU_PHASE4
ADT Phase
NUMERIC
False
B2 Vertical ADT Phase 4 Mon1 (deg)
132
ADTH.SR4.B1:TRANSVERSEACTIVITYMAX_HB1
ADT Activity
NUMERIC
False
B1 Horizontal ADT Transverse Activity (um)
133
ADTV.SR4.B1:TRANSVERSEACTIVITYMAX_VB1
ADT Activity
NUMERIC
False
B1 Vertical ADT Transverse Activity (um)
134
ADTH.SR4.B2:TRANSVERSEACTIVITYMAX_HB2
ADT Activity
NUMERIC
False
B2 Horizontal ADT Transverse Activity (um)
135
ADTV.SR4.B2:TRANSVERSEACTIVITYMAX_VB2
ADT Activity
NUMERIC
False
B2 Vertical ADT Transverse Activity (um)
136
LHC.BOFSU:DEFLECTIONS_H
Orbit
VECTORNUMERIC
False
COD deflections in horizontal plane (urad)
137
LHC.BOFSU:DEFLECTIONS_V
Orbit
VECTORNUMERIC
False
COD deflections in vertical plane (urad)
138
LHC.BPTUH.A4L5.B1:CALIBLINEARPOS
Orbit
NUMERIC
False
Upstream Horizontal TCT.A4L5.B1 Calibrated Lin...
139
LHC.BPTUV.A4L5.B1:CALIBLINEARPOS
Orbit
NUMERIC
False
Upstream Vertical TCT.A4L5.B1 Calibrated Linea...
140
LHC.BPTUH.A4R5.B2:CALIBLINEARPOS
Orbit
NUMERIC
False
Upstream Horizontal TCT.A4R5.B2 Calibrated Lin...
141
LHC.BPTUV.A4R5.B2:CALIBLINEARPOS
Orbit
NUMERIC
False
Upstream Vertical TCT.A4R5.B2 Calibrated Linea...
142
LHC.BPTDH.A4L5.B1:CALIBLINEARPOS
Orbit
NUMERIC
False
Downstream Horizontal TCT.A4L5.B1 Calibrated L...
143
LHC.BPTDV.A4L5.B1:CALIBLINEARPOS
Orbit
NUMERIC
False
Downstream Vertical TCT.A4L5.B1 Calibrated Lin...
144
LHC.BPTDH.A4R5.B2:CALIBLINEARPOS
Orbit
NUMERIC
False
Downstream Horizontal TCT.A4R5.B2 Calibrated L...
145
LHC.BPTDV.A4R5.B2:CALIBLINEARPOS
Orbit
NUMERIC
False
Downstream Vertical TCT.A4R5.B2 Calibrated Lin...
146
LHC.BPTUH.A4L1.B1:CALIBLINEARPOS
Orbit
NUMERIC
False
Upstream Horizontal TCT.A4L1.B1 Calibrated Lin...
147
LHC.BPTUV.A4L1.B1:CALIBLINEARPOS
Orbit
NUMERIC
False
Upstream Vertical TCT.A4L1.B1 Calibrated Linea...
148
LHC.BPTUH.A4R1.B2:CALIBLINEARPOS
Orbit
NUMERIC
False
Upstream Horizontal TCT.A4R1.B2 Calibrated Lin...
149
LHC.BPTUV.A4R1.B2:CALIBLINEARPOS
Orbit
NUMERIC
False
Upstream Vertical TCT.A4R1.B2 Calibrated Linea...
150
LHC.BPTDH.A4L1.B1:CALIBLINEARPOS
Orbit
NUMERIC
False
Downstream Horizontal TCT.A4L1.B1 Calibrated L...
151
LHC.BPTDV.A4L1.B1:CALIBLINEARPOS
Orbit
NUMERIC
False
Downstream Vertical TCT.A4L1.B1 Calibrated Lin...
152
LHC.BPTDH.A4R1.B2:CALIBLINEARPOS
Orbit
NUMERIC
False
Downstream Horizontal TCT.A4R1.B2 Calibrated L...
153
LHC.BPTDV.A4R1.B2:CALIBLINEARPOS
Orbit
NUMERIC
False
Downstream Vertical TCT.A4R1.B2 Calibrated Lin...
154 rows × 5 columns
LPC website
A rich source of information is the LPC website .
We will show how to import its data in pd dataframes.
Getting the filling table
def LPCFillTable ( year = 2018 ):
'''
It return the LPC Fill Table of the specified year.
Note that the fill table is available only for a limited number of year.
'''
import urllib.request , json
with urllib . request . urlopen ( 'http://lpc.web.cern.ch/cgi-bin/fillTableReader.py?action=load&year=' + str ( year )) as url :
data = json . loads ( url . read () . decode ())
if len ( data [ 'data' ]) > 0 :
myDF = pd . DataFrame ( data [ 'data' ]) . transpose ()[[ 'start_sb' , 'length_sb' , 'type' , 'scheme' , 'end' , 'ta' , 'fl' , 'remarks' ]]
return myDF . sort_index ()
else :
return pd . DataFrame ()
start_sb
length_sb
type
scheme
end
ta
fl
remarks
6469
2018-03-22 18:47:32
0:50:55
special
alternating R1 R2 pilot
operator
True
True
handshake tests
6570
2018-04-17 12:54:14
6:36:36
rampup
Single_3b_2_2_2
operator
True
True
First STABLE BEAMS in 2018. At the start befor...
6573
2018-04-18 00:29:29
6:20:49
rampup
Single_3b_2_2_2
operator
True
True
Second fill with crossing angle leveling and s...
6574
2018-04-18 22:28:06
11:33:43
rampup
Single_12b_8_8_8_2018
operator
True
True
Last fill of first intensity step. At the end ...
6579
2018-04-20 00:22:38
11:58:16
rampup
25ns_75b_62_32_62_12bpi_9inj
RF
True
True
First fill with trains. At the end ß* leveling...
6583
2018-04-20 19:48:23
5:58:46
rampup
25ns_75b_62_32_62_12bpi_9inj
others
True
True
Dump by COD interlock. Lumi was lower since th...
6584
2018-04-21 04:04:20
6:50:59
rampup
no_value
operator
True
True
ß* leveling in SB. Lumi in LHCb moved the same...
6592
2018-04-21 23:15:46
2:54:31
rampup
Single_10b_9_2_2_BSRT_Calib_RampUp
operator
True
True
Emittance between 1.5 and 4.5 µm
6594
2018-04-22 09:29:22
9:14:28
rampup
25ns_339b_326_206_266_12bpi_31inj
operator
True
True
Was stopped due to access request of ATLAS for...
6595
2018-04-22 22:14:52
9:04:39
rampup
25ns_339b_326_206_266_12bpi_31inj
operator
True
True
CMS took some low µ data. After this fill one ...
6611
2018-04-24 22:35:27
9:25:22
rampup
25ns_315b_302_237_240_48bpi_11inj
operator
True
True
Loss spike due to 16L2 reached 40% during ramp...
6612
2018-04-25 14:16:59
8:36:17
rampup
25ns_603b_590_524_542_48bpi_17inj
operator
True
True
6613
2018-04-26 01:23:52
0:29:22
rampup
25ns_603b_590_524_542_48bpi_17inj
others
True
True
BTV interlock. Crate exchanged
6614
2018-04-26 03:46:32
5:24:46
rampup
25ns_603b_590_524_542_48bpi_17inj
pwr converter
True
True
RCBCVS5.R2B1
6615
2018-04-26 19:54:06
5:11:26
rampup
25ns_603b_590_526_547_96bpi_13inj
operator
True
True
operator
6616
2018-04-27 04:44:15
8:10:27
rampup
25ns_987b_974_876_912_96bpi_17inj
operator
True
True
ALICE lower solenoid field (12kA instead of 30kA)
6617
2018-04-27 16:40:40
8:40:53
rampup
25ns_987b_974_878_917_144bpi_13inj
operator
False
True
During first filling trial one bunch was missi...
6618
2018-04-28 05:05:13
4:03:42
rampup
25ns_987b_974_878_917_144bpi_13inj
operator
True
True
6620
2018-04-28 21:31:28
5:12:16
rampup
25ns_1227b_1214_1054_1102_144bpi_14inj
16L2
True
True
...number one... Beam 2 dumped. ADT B1 H was off
6621
2018-04-29 05:17:00
10:19:29
rampup
25ns_1227b_1214_1054_1102_144bpi_14inj
operator
False
True
IP2 IP8 lumi was changing when crossing angle ...
6624
2018-04-30 01:52:55
11:12:05
rampup
25ns_1227b_1214_1054_1102_144bpi_14inj
operator
False
True
6628
2018-04-30 21:07:28
12:01:33
physics
25ns_1551b_1538_1404_1467_144bpi_16inj
operator
False
True
L=1.3e34; first fill with continuous crossing ...
6629
2018-05-01 12:07:32
10:38:32
physics
25ns_1887b_1874_1694_1772_144bpi_19inj
pwr converter
False
True
L=1.6e34; trip of orbit corrector; afterwards ...
6633
2018-05-02 06:08:50
0:47:31
physics
25ns_1887b_1874_1694_1772_144bpi_19inj
RF
False
True
During filling the first time uncaptured beam ...
6636
2018-05-02 11:41:44
2:41:30
physics
25ns_1887b_1874_1694_1772_144bpi_19inj
pwr converter
False
False
Orbit Corrector trip; communication fault.&nbs...
6638
2018-05-02 23:30:33
14:20:52
physics
25ns_2175b_2162_1874_1964_144bpi_19inj
operator
False
False
Abort gap keeper changed to 32811. RF voltage ...
6639
2018-05-03 16:57:34
11:54:41
physics
25ns_2319b_2306_1964_2060_144bpi_22inj
pwr converter
False
False
Lower currents since settings in Injectors wer...
6640
2018-05-04 10:07:21
9:09:26
physics
25ns_2460b_2448_2052_2154_144bpi_19injv2
operator
False
False
Lower intenity due to PS RF issue.
6641
2018-05-04 21:34:11
7:06:39
physics
25ns_2460b_2448_2052_2154_144bpi_19injv2
QPS
False
False
trip of RCBXH3.L5 and RCBXH3.L5 triggered by Q...
6642
2018-05-05 07:07:09
3:56:27
physics
25ns_2460b_2448_2052_2154_144bpi_19injv2
others
False
False
BLM dumped the beam (SUE candidarw)
...
...
...
...
...
...
...
...
...
7449
2018-11-16 18:31:13
7:23:15
ions
100_150ns_648Pb_620_619_52_36bpi_20inj_V2
operator
False
False
7450
2018-11-17 04:30:21
7:20:12
ions
100_150ns_648Pb_620_619_52_36bpi_20inj_V2
operator
False
False
After wards fill 7451 was again lost due to lo...
7453
2018-11-18 07:36:17
11:14:01
ions
100_150ns_648Pb_620_619_52_36bpi_20inj_V2
operator
False
False
kept long due to Linac-III source and LEIR pro...
7454
2018-11-19 00:35:24
11:21:55
ions
100_150ns_648Pb_620_619_52_36bpi_20inj_V2
operator
False
False
7456
2018-11-19 22:21:10
6:51:04
ions
100_150ns_648Pb_620_619_52_36bpi_20inj_V2
operator
False
False
7457
2018-11-20 11:03:57
7:16:12
ions
100_150ns_648Pb_620_619_52_36bpi_20inj_V2
operator
False
False
7459
2018-11-21 00:09:47
0:09:48
ions
75_150ns_460Pb_460_456_304_36bpi_17inj
operator
False
False
First ramp up fill with reversed polarity in A...
7460
2018-11-21 02:41:29
4:51:48
ions
75_150ns_460Pb_460_456_304_36bpi_17inj
operator
False
False
Second ramp up fill with reversed polarity in ...
7464
2018-11-22 03:58:13
6:31:19
ions
75_150ns_460Pb_460_456_304_42bpi_13inj
operator
False
False
Third ramp up fill.
7466
2018-11-22 19:08:59
8:37:46
ions
75_150ns_670Pb_607_630_384_42bpi_18inj
operator
False
False
Levelling IP 1&5 at 3.5e27. Reduced fillin...
7467
2018-11-23 09:58:59
7:58:52
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
operator
False
False
Levelling IP 1&5 at 4e27.
7468
2018-11-23 20:43:11
9:50:12
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
operator
False
False
Levelling IP 1&5 at 5e27.
7471
2018-11-24 14:35:23
8:07:52
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
operator
False
False
7472
2018-11-25 01:25:01
7:50:12
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
operator
False
False
7473
2018-11-25 11:49:05
9:48:03
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
operator
False
False
Problem with inner detector power supply in AL...
7474
2018-11-26 02:13:25
5:55:26
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
pwr converter
False
False
Power converters losing communication
7475
2018-11-26 18:06:32
0:38:36
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
QPS
False
False
Spurious QPS trip in RB of S12
7477
2018-11-26 23:27:29
8:50:01
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
operator
False
False
7480
2018-11-28 00:10:18
7:53:18
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
operator
False
False
7481
2018-11-28 11:00:42
8:23:53
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
operator
False
False
7482
2018-11-28 23:10:05
0:31:41
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
operator
False
False
Losses in IR7 with 10Hz oscillations
7483
2018-11-29 02:33:11
13:01:29
ions
100_150ns_648Pb_620_619_52_36bpi_20inj_V2
operator
False
False
VDM repetition for ALICE in shadow of
7485
2018-11-29 21:29:08
7:51:20
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
operator
False
False
Slow filling
7486
2018-11-30 09:38:54
7:35:52
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
operator
False
False
7487
2018-11-30 20:04:57
8:29:44
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
operator
False
False
7488
2018-12-01 07:40:42
8:05:50
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
operator
False
False
7489
2018-12-01 18:27:18
0:44:40
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
others
False
False
Vacuum
7490
2018-12-01 21:37:48
8:31:35
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
operator
False
False
7491
2018-12-02 08:48:41
4:50:14
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
RF
False
False
RF
7492
2018-12-02 15:52:49
1:18:06
ions
75_150ns_733Pb_733_702_468_42bpi_20inj
operator
False
False
291 rows × 8 columns
Luminosity data
def LPCLumiData ( source = 'http://lpc-afs.web.cern.ch/lpc-afs/LHC/2018/luminosity_data_ions.txt' ):
import requests
response = requests . get ( 'http://lpc-afs.web.cern.ch/lpc-afs/LHC/2018/luminosity_data_ions.txt' )
data = response . text
aux = ( pd . DataFrame ([ x . split () for x in data . split ( ' \n ' )[ 2 :]], columns = [ 'fill' , 'tstart_UNIX_UTC' , 'tstop_UNIX_UTC' , 'ATLAS_lui_inverse_ub' , \
'CMS_lui_inverse_ub' , 'LHCb_lui_inverse_ub' , 'ALICE_lui_inverse_ub' , 'ATLAS_plu_Hz_inverse_ub' , \
'CMS_plu_Hz_inverse_ub' , 'LHCb_plu_Hz_inverse_ub' , 'ALICE_plu_Hz_inverse_ub' ]))
aux = aux . set_index ( 'fill' )
aux . index . name = None
return aux
LPCLumiData ( source = 'http://lpc-afs.web.cern.ch/lpc-afs/LHC/2018/luminosity_data_ions.txt' )
tstart_UNIX_UTC
tstop_UNIX_UTC
ATLAS_lui_inverse_ub
CMS_lui_inverse_ub
LHCb_lui_inverse_ub
ALICE_lui_inverse_ub
ATLAS_plu_Hz_inverse_ub
CMS_plu_Hz_inverse_ub
LHCb_plu_Hz_inverse_ub
ALICE_plu_Hz_inverse_ub
7427
1541708363
1541733698
1
1
0
2
1.510e-04
1.720e-04
3.200e-05
2.065e-04
7428
1541741102
1541752703
2
2
0
2
1.320e-04
1.290e-04
2.900e-05
8.785e-05
7433
1541805329
1541820593
10
10
0
6
8.220e-04
8.150e-04
2.800e-05
4.479e-04
7435
1541864777
1541885107
21
21
1
12
9.940e-04
9.870e-04
3.000e-05
5.904e-04
7436
1541897140
1541917561
46
47
2
26
2.461e-03
2.445e-03
8.700e-05
9.689e-04
7437
1541933644
1541947812
64
64
2
34
2.350e-03
2.360e-03
7.600e-05
9.662e-04
7438
1541962710
1541986715
89
89
5
54
2.043e-03
2.016e-03
1.970e-04
1.002e-03
7439
1542004896
1542030234
125
125
6
75
2.999e-03
2.960e-03
1.220e-04
1.044e-03
7440
1542049336
1542077104
163
162
9
96
3.075e-03
3.775e-03
1.660e-04
1.172e-03
7441
1542090535
1542117430
179
169
13
114
1.443e-03
1.459e-03
2.010e-04
9.124e-04
7442
1542135376
1542149279
185
181
15
126
2.068e-03
2.059e-03
2.710e-04
9.953e-04
7443
1542212137
1542233763
192
195
19
144
1.843e-03
1.845e-03
2.390e-04
9.750e-04
7444
1542247020
1542273588
235
238
20
166
2.223e-03
2.231e-03
1.000e-04
1.021e-03
7446
1542309239
1542335749
279
283
22
189
2.249e-03
2.231e-03
9.300e-05
1.012e-03
7447
1542344205
1542344424
279
283
22
189
0.000e+00
1.888e-03
1.600e-04
1.018e-03
7448
1542353526
1542380442
323
327
23
211
2.248e-03
2.660e-03
1.030e-04
1.030e-03
7449
1542389473
1542416067
369
373
25
235
2.261e-03
2.274e-03
1.030e-04
1.050e-03
7450
1542425421
1542451833
412
416
26
257
2.238e-03
2.231e-03
1.010e-04
1.044e-03
7453
1542522977
1542568209
467
472
28
287
2.228e-03
2.274e-03
1.190e-04
1.058e-03
7454
1542584123
1542642529
513
517
30
315
2.231e-03
2.231e-03
8.900e-05
1.121e-02
7456
1542662470
1542687133
555
559
32
340
2.235e-03
2.231e-03
1.020e-04
1.075e-03
7457
1542708236
1542734526
598
602
33
362
2.239e-03
2.231e-03
1.000e-04
1.040e-03
7459
1542755386
1542755976
599
604
33
363
2.720e-03
2.746e-03
5.670e-04
1.016e-03
7460
1542764489
1542781997
633
638
39
379
2.751e-03
2.703e-03
5.700e-04
1.041e-03
7464
1542855492
1542878972
671
677
46
399
2.713e-03
2.703e-03
5.560e-04
1.220e-03
7466
1542910139
1542941206
727
734
55
429
3.522e-03
3.518e-03
5.240e-04
1.087e-03
7467
1542963538
1542992270
791
799
67
458
4.030e-03
4.033e-03
1.205e-03
1.073e-03
7468
1543002191
1543037603
866
875
79
492
5.142e-03
5.148e-03
1.014e-03
1.071e-03
7471
1543066522
1543095796
932
942
91
521
4.995e-03
4.976e-03
9.950e-04
1.102e-03
7472
1543105500
1543133713
998
1009
102
549
5.007e-03
4.976e-03
1.025e-03
1.132e-03
7473
1543142944
1543178229
1074
1085
115
588
6.125e-03
6.092e-03
1.057e-03
1.290e-03
7474
1543194804
1543216131
1133
1144
125
609
5.917e-03
5.834e-03
1.019e-03
1.070e-03
7475
1543251991
1543254308
1143
1147
127
611
6.107e-03
5.577e-03
1.020e-03
1.065e-03
7477
1543271249
1543303184
1210
1214
139
642
5.648e-03
5.663e-03
1.000e-03
1.067e-03
7480
1543360218
1543388616
1278
1285
150
671
5.986e-03
5.963e-03
1.098e-03
1.084e-03
7481
1543399242
1543429605
1351
1360
163
701
5.969e-03
6.006e-03
1.037e-03
1.093e-03
7482
1543443005
1543444906
1360
1368
165
702
5.721e-03
5.577e-03
1.025e-03
1.072e-03
7483
1543455191
1543502081
1409
1415
167
738
4.059e-03
3.947e-03
2.760e-04
1.059e-03
7485
1543523347
1543551628
1466
1471
177
766
4.610e-03
4.547e-03
9.350e-04
1.066e-03
7486
1543567133
1543594487
1522
1527
187
794
4.836e-03
4.633e-03
9.500e-04
1.295e-03
7487
1543604697
1543635280
1588
1594
198
825
5.600e-03
5.620e-03
1.065e-03
1.064e-03
7488
1543646442
1543675593
1655
1660
210
854
5.836e-03
5.749e-03
1.017e-03
1.074e-03
7489
1543685238
1543687919
1668
1673
212
856
5.787e-03
5.792e-03
9.990e-04
1.059e-03
7490
1543696668
1543727362
1736
1741
224
887
5.887e-03
5.834e-03
1.006e-03
1.085e-03
7491
1543736920
1543754334
1788
1794
234
904
5.798e-03
5.749e-03
1.002e-03
1.075e-03
7492
1543762369
1543767055
1797
1802
235
907
2.343e-03
2.231e-03
2.470e-04
7.774e-04
NaN
None
None
None
None
None
None
None
None
None
None
The instability table
def LHCInstabilityTable ():
import requests
response = requests . get ( 'https://lhcinstability.web.cern.ch/lhcinstability/FillSummaries/2018/index.html' )
data = response . text
aux = [ x . split ( '</a>' ) for x in data . split ( '<a href="' )]
aux = aux [ 1 :]
fill = [ int ( x [ 0 ] . split ( '/' )[ 0 ]) for x in aux ]
comment = [ x [ 1 ] . split ( '<!--PLACEHOLDER-->' )[ 0 ] for x in aux ]
aux = pd . DataFrame ({ 'fill' : fill , 'Comment' : comment })
aux = aux . set_index ( 'fill' )
aux . index . name = None
return aux
Comment
6484
- No instability detected.<br>\n
6492
- No instability detected.<br>\n
6493
- No instability detected.<br>\n
6494
- No instability detected.<br>\n
6495
- No instability detected.<br>\n
6496
\n - Instabilities of probe bunches during the...
6497
- No instability detected.<br>\n
6498
- No instability detected.<br>\n
6499
- No instability detected.<br>\n
6500
- No instability detected.<br>\n
6501
- No instability detected.<br>\n
6502
- No instability detected.<br>\n
6503
- No instability detected.<br>\n
6504
- No instability detected.<br>\n
6505
- No instability detected.<br>\n
6506
- No instability detected.<br>\n
6507
- No instability detected.<br>\n
6508
- No instability detected.<br>\n
6509
- No instability detected.<br>\n
6510
- No instability detected.<br>\n
6511
- No instability detected.<br>\n
6512
- No instability detected.<br>\n
6513
- No instability detected.<br>\n
6514
- No instability detected.<br>\n
6516
- No instability detected.<br>\n
6517
- No instability detected.<br>\n
6518
- No instability detected.<br>\n
6519
- No instability detected.<br>\n
6520
- No instability detected.<br>\n
6521
- No instability detected.<br>\n
...
...
7462
- Loss maps at injection and flat-top with 20...
7463
- No instability detected. Validation cycle w...
7464
- No instability detected.<br>\n
7465
- No coherent instability detected. Dumped by...
7466
- No instability detected.<br>\n
7467
- No instability detected.<br>\n
7468
- No instability detected.<br>\n
7469
- No beam (only probes).<br>\n
7470
- RF resynchronization first. Then beam was r...
7471
- No instability detected.<br>\n
7472
- No instability detected.<br>\n
7473
- No instability detected. Activity in V for ...
7474
- No instability detected.<br>\n
7475
- No instability detected.<br>\n
7477
- No instability detected.<br>\n
7478
- Crystal collimation MD.<br>\n
7479
- Crystal collimation MD.<br>\n
7480
- No instability detected.<br>\n
7481
- no emittance blow-up. Vertical activity at ...
7482
- beams dumped by 10 Hz oscillations 30 minut...
7483
- no emittance blow-up. Vertical activity at ...
7485
- no emittance blow-up. Vertical activity at ...
7486
- no emittance blow-up. Vertical activity at ...
7487
- no emittance blow-up. Vertical activity at ...
7488
- no emittance blow-up. Vertical activity at ...
7489
- no emittance blow-up. Vertical activity at ...
7490
- no emittance blow-up. Vertical activity at ...
7491
- no emittance blow-up. Vertical activity at ...
7492
- no emittance blow-up. Vertical activity at ...
7493
- optics MD. <br>\n
893 rows × 1 columns
Building a fill table
import numpy as np
listDF = []
myFills = np . arange ( 6666 , 6670 )
for i in np . unique ( myFills ):
aux = importData . LHCFillsByNumber ( i )
myDF = aux [ aux [ 'mode' ] == 'FILL' ] . copy ()
del myDF [ 'mode' ]
myDF [ 'modes DF' ] = [ aux [ ~ ( aux [ 'mode' ] == 'FILL' )]]
aux = myDF . iloc [ 0 ][ 'modes DF' ] . groupby ( 'mode' ) . duration . sum ()
aux = pd . DataFrame ( aux ) . transpose ()
aux . columns . name = None
aux [ 'fill' ] = i
aux = aux . set_index ( 'fill' )
aux . index . name = None
#aux1=importData.LHCCals2pd(['LHC.STATS:B%_NUMBER_BUNCHES'],fillList=[i])
listDF . append ( pd . concat ([ aux , myDF ], axis = 1 ))
fillsDF = pd . concat ( listDF , sort = True )
This is the minimal fill DF contatain the specified fill and the relative duration modes.
ADJUST
BEAMDUMP
FLATTOP
INJPHYS
INJPROT
PRERAMP
RAMP
RAMPDOWN
SETUP
SQUEEZE
STABLE
duration
endTime
modes DF
startTime
6666
00:32:01.453000
00:02:24.449000
00:02:33.174000
00:45:26.077000
00:03:48.286999
00:08:49.490999
00:20:14.409999
00:04:21.061000
NaT
00:10:54.578999
12:43:12.336000
14:53:58.320000
2018-05-11 10:29:36.956000090+00:00
mode start...
2018-05-10 19:35:38.635999918+00:00
6667
NaT
NaT
NaT
00:43:23.439999
00:10:15.817999
NaT
NaT
NaT
00:07:09.145999
NaT
NaT
01:35:39.635999
2018-05-11 12:05:16.592999935+00:00
mode startT...
2018-05-11 10:29:36.957000017+00:00
6668
NaT
NaT
NaT
00:10:28.993000
00:04:30.044999
NaT
NaT
NaT
NaT
NaT
NaT
00:15:09.246999
2018-05-11 12:20:25.841000080+00:00
mode startT...
2018-05-11 12:05:16.594000101+00:00
6669
NaT
NaT
NaT
00:10:24.634999
00:15:45.363000
NaT
NaT
NaT
NaT
NaT
NaT
00:29:17.625000
2018-05-11 12:49:43.467000008+00:00
mode startT...
2018-05-11 12:20:25.842000008+00:00
fillsDF [( fillsDF [ 'STABLE' ] > pd . Timedelta ( '10h' ))]
ADJUST
BEAMDUMP
FLATTOP
INJPHYS
INJPROT
PRERAMP
RAMP
RAMPDOWN
SETUP
SQUEEZE
STABLE
duration
endTime
modes DF
startTime
6666
00:32:01.453000
00:02:24.449000
00:02:33.174000
00:45:26.077000
00:03:48.286999
00:08:49.490999
00:20:14.409999
00:04:21.061000
NaT
00:10:54.578999
12:43:12.336000
14:53:58.320000
2018-05-11 10:29:36.956000090+00:00
mode start...
2018-05-10 19:35:38.635999918+00:00
importData . LHCFillsAggregation ([ 'LHC.BCTFR.A6R4.B%:BEAM_INTENSITY' , 'LHC.BQM.B%:NO_BUNCHES' , 'LHC.BQM.B1:NO_BUNCHES' ], 6666 , [ 'INJPHYS' ], functionList = np . max , mapInsteadAgg = False )
LHC.BCTFR.A6R4.B1:BEAM_INTENSITY
LHC.BCTFR.A6R4.B2:BEAM_INTENSITY
LHC.BQM.B1:NO_BUNCHES
LHC.BQM.B2:NO_BUNCHES
startTime
endTime
duration
fill
mode
6666
INJPHYS
2.833537e+14
2.856917e+14
2556.0
2556.0
2018-05-10 19:39:39.915999889+00:00
2018-05-10 20:25:05.993000031+00:00
00:45:26.077000
importData . LHCFillsAggregation ([ 'LHC.BQM.B%:NO_BUNCHES' ], range ( 6666 , 6670 ), functionList = np . max , mapInsteadAgg = False )
LHC.BQM.B1:NO_BUNCHES
LHC.BQM.B2:NO_BUNCHES
startTime
endTime
duration
fill
mode
6666
FILL
2556.0
2556.0
2018-05-10 19:35:38.635999918+00:00
2018-05-11 10:29:36.956000090+00:00
14:53:58.320000
6667
FILL
2.0
2.0
2018-05-11 10:29:36.957000017+00:00
2018-05-11 12:05:16.592999935+00:00
01:35:39.635999
6668
FILL
3.0
2.0
2018-05-11 12:05:16.594000101+00:00
2018-05-11 12:20:25.841000080+00:00
00:15:09.246999
6669
FILL
12.0
1.0
2018-05-11 12:20:25.842000008+00:00
2018-05-11 12:49:43.467000008+00:00
00:29:17.625000
['LHC.BSRA.US45.B1/XPocAcquisition#aGXpocNumberAcq',
'LHC.BSRA.US45.B2/XPocAcquisition#aGXpocNumberAcq',
'LHC.BSRT.5L4.B2:GATENUMBEROFPULSES',
'LHC.BSRT.5R4.B1:GATENUMBEROFPULSES',
'LHC.STATS:B1_NUMBER_BUNCHES',
'LHC.STATS:B2_NUMBER_BUNCHES',
'LHC.STATS:NUMBER_COLLISIONS_IP1_5',
'LHC.STATS:NUMBER_COLLISIONS_IP2',
'LHC.STATS:NUMBER_COLLISIONS_IP8',
'LHC.STATS:NUMBER_OF_INJECTIONS',
'LMFGLO2_STATUS_FLUSHING_NUMBER.POSST',
'LMFGLO3_STATUS_FLUSHING_NUMBER.POSST',
'LP.NSRCGEN:NUMBEROFSPARKS',
'SIMA.10L3.3RM10S:REV_NUMBER_OBSOLETE',
'SIMA.10L7.7LM08S:REV_NUMBER_OBSOLETE',
'SIMA.10R3.3RM20S:REV_NUMBER_OBSOLETE',
'SIMA.10R7.7RM10S:REV_NUMBER_OBSOLETE',
'SIMA.11L3.3RM09S:REV_NUMBER_OBSOLETE',
'SIMA.11L7.7LM09S:REV_NUMBER_OBSOLETE',
'SIMA.11R3.3RM21S:REV_NUMBER_OBSOLETE',
'SIMA.11R7.7RM11S:REV_NUMBER_OBSOLETE',
'SIMA.12L3.3RM08S:REV_NUMBER_OBSOLETE',
'SIMA.12L7.7LM10S:REV_NUMBER_OBSOLETE',
'SIMA.12R3.3RM22S:REV_NUMBER_OBSOLETE',
'SIMA.12R7.7RM12S:REV_NUMBER_OBSOLETE',
'SIMA.13L3.3RM07S:REV_NUMBER_OBSOLETE',
'SIMA.13L7.7LM11S:REV_NUMBER_OBSOLETE',
'SIMA.13R3.3RM23S:REV_NUMBER_OBSOLETE',
'SIMA.13R7.7RM13S:REV_NUMBER_OBSOLETE',
'SIMA.14L3.3RM06S:REV_NUMBER_OBSOLETE',
'SIMA.14L7.7LM12S:REV_NUMBER_OBSOLETE',
'SIMA.14R3.3RM24S:REV_NUMBER_OBSOLETE',
'SIMA.14R7.7RM14S:REV_NUMBER_OBSOLETE',
'SIMA.15L3.3RM05S:REV_NUMBER_OBSOLETE',
'SIMA.15L7.7LM13S:REV_NUMBER_OBSOLETE',
'SIMA.15R3.3RM25S:REV_NUMBER_OBSOLETE',
'SIMA.15R7.7RM15S:REV_NUMBER_OBSOLETE',
'SIMA.16L3.3RM04S:REV_NUMBER_OBSOLETE',
'SIMA.16L7.7LM14S:REV_NUMBER_OBSOLETE',
'SIMA.16R3.3RM26S:REV_NUMBER_OBSOLETE',
'SIMA.16R7.7RM16S:REV_NUMBER_OBSOLETE',
'SIMA.17L3.3RM03S:REV_NUMBER_OBSOLETE',
'SIMA.17L7.7LM15S:REV_NUMBER_OBSOLETE',
'SIMA.17R3.3RM27S:REV_NUMBER_OBSOLETE',
'SIMA.17R7.7RM17S:REV_NUMBER_OBSOLETE',
'SIMA.18L7.7LM16S:REV_NUMBER_OBSOLETE',
'SIMA.18R3.3RM28S:REV_NUMBER_OBSOLETE',
'SIMA.18R7.7RM18S:REV_NUMBER_OBSOLETE',
'SIMA.19L7.7LM17S:REV_NUMBER_OBSOLETE',
'SIMA.19R3.3RM29S:REV_NUMBER_OBSOLETE',
'SIMA.19R7.7RM19S:REV_NUMBER_OBSOLETE',
'SIMA.20L7.7LM18S:REV_NUMBER_OBSOLETE',
'SIMA.20R3.3RM30S:REV_NUMBER_OBSOLETE',
'SIMA.20R7.7RM20S:REV_NUMBER_OBSOLETE',
'SIMA.21R7.7RM21S:REV_NUMBER_OBSOLETE',
'SIMA.4L3.3RM17S:REV_NUMBER_OBSOLETE',
'SIMA.4L7.7LM01S:REV_NUMBER_OBSOLETE',
'SIMA.4R7.7RM03S:REV_NUMBER_OBSOLETE',
'SIMA.5L3.3RM16S:REV_NUMBER_OBSOLETE',
'SIMA.6L3.3RM15S:REV_NUMBER_OBSOLETE',
'SIMA.8L3.3RM12S:REV_NUMBER_OBSOLETE',
'SIMA.8L7.7LM06S:REV_NUMBER_OBSOLETE',
'SIMA.8R3.3RM18S:REV_NUMBER_OBSOLETE',
'SIMA.8R7.7RM08S:REV_NUMBER_OBSOLETE',
'SIMA.9L3.3RM11S:REV_NUMBER_OBSOLETE',
'SIMA.9L7.7LM07S:REV_NUMBER_OBSOLETE',
'SIMA.9R3.3RM19S:REV_NUMBER_OBSOLETE',
'SIMA.9R7.7RM09S:REV_NUMBER_OBSOLETE',
'SIMA.RE38.3RM31S:REV_NUMBER_OBSOLETE',
'SIMA.RE38.3RM32S:REV_NUMBER_OBSOLETE',
'SIMA.RE72.7LM19S:REV_NUMBER_OBSOLETE',
'SIMA.RE72.7LM20S:REV_NUMBER_OBSOLETE',
'SIMA.RE78.7RM22S:REV_NUMBER_OBSOLETE',
'SIMA.RE78.7RM23S:REV_NUMBER_OBSOLETE',
'SIMA.RR73.7LM05S:REV_NUMBER_OBSOLETE',
'SIMA.RR77.7RM07S:REV_NUMBER_OBSOLETE',
'SIMA.UJ33.3RM13S:REV_NUMBER_OBSOLETE',
'SIMA.UJ33.3RM14S:REV_NUMBER_OBSOLETE',
'SIMA.UJ76.7RM01S:REV_NUMBER_OBSOLETE',
'SPS.BWS.41677.V_ROT.APP.IN:BU_NUMBERS',
'SPS.BWS.41677.V_ROT.APP.OUT:BU_NUMBERS',
'SPS.BWS.51995.H_ROT.APP.IN:BU_NUMBERS',
'SPS.BWS.51995.H_ROT.APP.OUT:BU_NUMBERS',
'SPSQC:LHC_NUMBER_OF_EXECUTED_BATCHES',
'SPSQC:NUMBER_OF_BUNCHES_FLAT_TOP',
'SPSQC:NUMBER_OF_INJECTIONS']