Nicer - Revision history

Thalhammer at 00:21, 24 June 2026

2026-06-24T00:21:28Z

← Older revision		Revision as of 02:21, 24 June 2026
Line 28:		Line 28:
	can apply additional filter criteria during the preparation. See		can apply additional filter criteria during the preparation. See
	'xtiprepare --help' for more information.		'xtiprepare --help' for more information.

			By default very short or heavily fragmented good-time intervals (GTIs) can be
			discarded during the level-2 screening. If you want to keep such short
			exposures, you can lower the shortest GTI window that is retained using the
			'--mingti' option (which is passed on to nicerl2). See also "Short and
			fragmented GTIs" below.


Line 49:		Line 55:
	subdirecory 'dirname' in $prepdir. Observation-specific		subdirecory 'dirname' in $prepdir. Observation-specific
	response files are generated and header keywords are set accordingly. It is still advisable to double-check that they are correctly loaded into e.g. ISIS.		response files are generated and header keywords are set accordingly. It is still advisable to double-check that they are correctly loaded into e.g. ISIS.

			The barycenter correction is not performed by default; add the '--bary' flag to
			enable it. The source position is taken from the event-file header unless you
			override it with '--ra' and '--dec'. <br />
			For observations with heavily fragmented or sub-bin GTIs the per-FPM
			normalization of the light-curve can fail. In that case run 'xtiextract' with
			'--nonorm', which disables the detector normalization (detnormtype=NONE) in
			nicerl3-lc. See also "Short and fragmented GTIs" below.

	=== Background generation: ===		=== Background generation: ===
Line 60:		Line 74:
	option to adjust/fit the different background components.		option to adjust/fit the different background components.

	You can switch between them using the ~~bkgmod_~~ parameter of extraction.		You can switch between them using the '--bkgmod_sp' parameter of extraction (for
	Only the sw-model is currently able to also provide background light-curves.		the spectrum, default 'scorpeon') and '--bkgmod_lc' (for the light-curve).
			Only the sw-model is currently able to also provide background light-curves.
	For more information see :<br />		For more information see :<br />
	https://heasarc.gsfc.nasa.gov/docs/nicer/tools/nicer_bkg_est_tools.html		https://heasarc.gsfc.nasa.gov/docs/nicer/tools/nicer_bkg_est_tools.html
Line 101:		Line 116:
	In that case one might want to increase the allowed range e.g. for		In that case one might want to increase the allowed range e.g. for
	"underonly_range" in "xtiprepare".		"underonly_range" in "xtiprepare".

			''' Short and fragmented GTIs: '''

			Some observations consist of many very short good-time intervals (GTIs), e.g.
			after aggressive filtering or for short pointings. Two problems can arise from
			this. First, GTIs shorter than the default minimum length are dropped during the
			preparation. Lower the retained GTI length with the '--mingti' option of
			'xtiprepare' (passed on to nicerl2) to keep them. Second, the per-FPM
			normalization applied by nicerl3-lc during the light-curve extraction can fail
			when the GTIs are heavily fragmented or shorter than the chosen time bin. In that
			case run 'xtiextract' with '--nonorm', which disables the detector normalization
			(detnormtype=NONE).

	''' Large event files: '''		''' Large event files: '''
Line 128:		Line 155:
	are unaffected.		are unaffected.



	[[Category:Data Extraction]]		[[Category:Data Extraction]]

Thalhammer: Update on RMF/ARF selection

2025-04-10T13:31:29Z

Update on RMF/ARF selection

← Older revision		Revision as of 15:31, 10 April 2025
Line 6:		Line 6:
	imaging capabilities. While this simplifies the data extraction, it is also required		imaging capabilities. While this simplifies the data extraction, it is also required
	to simulate a background for each observation. <br /> Previously downloaded		to simulate a background for each observation. <br /> Previously downloaded
	NICER observations can be found under /~~satdata~~/X-ray/nicer/.		NICER observations can be found under /data/X-ray/nicer/.


Line 47:		Line 47:

	The produced spectrum and light-curve can then be found in the		The produced spectrum and light-curve can then be found in the
	subdirecory 'dirname' in $prepdir. ~~Currently no observation~~-specific		subdirecory 'dirname' in $prepdir. Observation-specific
	response files are generated~~, but instead soft links~~ to ~~the current ARF~~		response files are generated and header keywords are set accordingly. It is still advisable to double-check that they are correctly loaded into e.g. ISIS.
	~~& RMF files included in the CALDB~~ are ~~generated~~.

	=== Background generation: ===		=== Background generation: ===
Line 62:		Line 61:

	You can switch between them using the bkgmod_ parameter of extraction.		You can switch between them using the bkgmod_ parameter of extraction.
	Only the sw-model is ~~curreltly~~ able to also provide background light-curves.		Only the sw-model is currently able to also provide background light-curves.
	For more information see :<br />		For more information see :<br />
	https://heasarc.gsfc.nasa.gov/docs/nicer/tools/nicer_bkg_est_tools.html		https://heasarc.gsfc.nasa.gov/docs/nicer/tools/nicer_bkg_est_tools.html

Thalhammer: /* Background generation: */

2023-06-07T10:34:07Z

Background generation:

← Older revision		Revision as of 12:34, 7 June 2023
Line 54:		Line 54:

	As we cannot produce a background spectrum from an off-source region on the		As we cannot produce a background spectrum from an off-source region on the
	detector, a background spectrum has to be modeled. Currently there are ~~two~~		detector, a background spectrum has to be modeled. Currently there are three
	different tools available for that purpose. One based on external parameters		different tools available for that purpose (sw', '3c50' and 'scorpeon').
	during the observation (i.e., space weather, solar angle and cutoff-rigidity of		One based on external parameters during the observation (i.e., space weather,
	earths B-field) ~~amd~~ one based on the internal distribution of measured events.		solar angle and cutoff-rigidity of earths B-field), one based on the internal
	~~Both are still in active development and might not be able to produce realistic~~		distribution of measured events and one using both and giving the user the
	~~spectra for all observations~~. ~~In that case a simple reference observation of one~~		option to adjust/fit the different background components.
	of the ~~RXTE~~ background ~~fields might be used instead~~. ~~<br />~~
	~~Both tools are further described on the instruments webpage~~:<br />		You can switch between them using the bkgmod_ parameter of extraction.
			Only the sw-model is curreltly able to also provide background light-curves.
			For more information see :<br />
	https://heasarc.gsfc.nasa.gov/docs/nicer/tools/nicer_bkg_est_tools.html		https://heasarc.gsfc.nasa.gov/docs/nicer/tools/nicer_bkg_est_tools.html



	=== Issues to keep in mind: ===		=== Issues to keep in mind: ===

Thalhammer at 10:09, 7 June 2023

2023-06-07T10:09:14Z

← Older revision		Revision as of 12:09, 7 June 2023
Line 121:		Line 121:

	''' Damage to the optical bench '''		''' Damage to the optical bench '''

	On May 22, 2023, between 13:00 and 14:00 UTC, NICER appears to have developed a		On May 22, 2023, between 13:00 and 14:00 UTC, NICER appears to have developed a
	visible-light leak in the X-ray Timing Instrument (XTI) optical bench. This leads to		visible-light leak in the X-ray Timing Instrument (XTI) optical bench. This leads to

Thalhammer at 10:06, 7 June 2023

2023-06-07T10:06:36Z

← Older revision		Revision as of 12:06, 7 June 2023
Line 95:		Line 95:
	observations.		observations.
	If that happens you should check the .mkf-file and see whether any of the parameters		If that happens you should check the .mkf-file and see whether any of the parameters
	exceeds the default filter criteria ~~sepcfied~~ in the fhelp of nimaketime.		exceeds the default filter criteria specified in the fhelp of nimaketime.
	This can be done e.g. using fplot:		This can be done e.g. using fplot:
	fplot *.mkf TIME "FPM_UNDERONLY_COUNT SUN_ANGLE FPM_OVERONLY_COUNT SUNSHINE "		fplot *.mkf TIME "FPM_UNDERONLY_COUNT SUN_ANGLE FPM_OVERONLY_COUNT SUNSHINE "
Line 120:		Line 120:
	More info here: https://heasarc.gsfc.nasa.gov/docs/nicer/analysis_threads/arf-rmf/		More info here: https://heasarc.gsfc.nasa.gov/docs/nicer/analysis_threads/arf-rmf/

			''' Damage to the optical bench '''
			On May 22, 2023, between 13:00 and 14:00 UTC, NICER appears to have developed a
			visible-light leak in the X-ray Timing Instrument (XTI) optical bench. This leads to
			and increase in the undershoot rate, background at soft energies and can lead to
			detectors switching off, telemetry saturation, and spurious timing features.
			This only affects observations performed during orbital days. Night time observations
			are unaffected.


	[[Category:Data Extraction]]		[[Category:Data Extraction]]

Thalhammer: /* Issues to keep in mind: */

2023-06-07T09:33:51Z

Issues to keep in mind:

← Older revision		Revision as of 11:33, 7 June 2023
Line 93:		Line 93:
	It can happen, that e.g. optical loading leads to a filter parameter		It can happen, that e.g. optical loading leads to a filter parameter
	exceeding the standard filter criteria for most of certain		exceeding the standard filter criteria for most of certain
	observations. Such an observation might still be of interest, however.		observations.
			If that happens you should check the .mkf-file and see whether any of the parameters
			exceeds the default filter criteria sepcfied in the fhelp of nimaketime.
			This can be done e.g. using fplot:
			fplot *.mkf TIME "FPM_UNDERONLY_COUNT SUN_ANGLE FPM_OVERONLY_COUNT SUNSHINE "

			Such an observation might still be of interest, however.
	In that case one might want to increase the allowed range e.g. for		In that case one might want to increase the allowed range e.g. for
	"underonly_range" in "xtiprepare".		"underonly_range" in "xtiprepare".

Berger at 09:20, 7 June 2023

2023-06-07T09:20:53Z

← Older revision		Revision as of 11:20, 7 June 2023
Line 19:		Line 19:
	#!/bin/csh		#!/bin/csh

	set datadir = /~~satdata~~/X-ray/nicer/data/2200570101		set datadir = /data/X-ray/nicer/data/2200570101
	set prepdir = '/userdata/data/$user/where/ever/you/want'		set prepdir = '/userdata/data/$user/where/ever/you/want'

Thalhammer at 01:19, 17 March 2021

2021-03-17T01:19:55Z

← Older revision		Revision as of 03:19, 17 March 2021
Line 113:		Line 113:
	active during the observation. <br />		active during the observation. <br />
	More info here: https://heasarc.gsfc.nasa.gov/docs/nicer/analysis_threads/arf-rmf/		More info here: https://heasarc.gsfc.nasa.gov/docs/nicer/analysis_threads/arf-rmf/


			[[Category:Data Extraction]]

Thalhammer: Wiki page for NICER data extraction

2021-03-17T01:17:31Z

Wiki page for NICER data extraction

New page

== NICER data extraction ==

NICER is a soft X-ray observatory on board the ISS. Biggest difference to many
other modern X-ray satellites are its silicon drift detectors and X-ray
concentrators (instead of focusing mirrors), which together do not permit
imaging capabilities. While this simplifies the data extraction, it is also required
to simulate a background for each observation. Previously downloaded
NICER observations can be found under /satdata/X-ray/nicer/.

=== Preparation:===

To apply the the latest calibration and filtering, first run
'xtiprepare'. The script will produce cleaned event
files inside the subdirectory 'reprocessed' inside 'prepdir'.

<pre>
#!/bin/csh

set datadir = /satdata/X-ray/nicer/data/2200570101
set prepdir = '/userdata/data/$user/where/ever/you/want'

xtiprepare --datadir=$datadir --prepdir=$prepdir
</pre>

If you notice an increased background in the resulting spectra, you
can apply additional filter criteria during the preparation. See
'xtiprepare --help' for more information.

=== Spectrum and Light-curve Extraction:===

To produce light-curves, spectra and apply barycenter correction
'xtiextract' can be used as follows.

<pre>
#!/bin/csh

set prepdir = '/userdata/data/$user/where/ever/you/want'
set dirname = 'extraction_123'

xtiextract --name=$dirname --prepdir=$prepdir

</pre>

The produced spectrum and light-curve can then be found in the
subdirecory 'dirname' in $prepdir. Currently no observation-specific
response files are generated, but instead soft links to the current ARF
& RMF files included in the CALDB are generated.

=== Background generation: ===

As we cannot produce a background spectrum from an off-source region on the
detector, a background spectrum has to be modeled. Currently there are two
different tools available for that purpose. One based on external parameters
during the observation (i.e., space weather, solar angle and cutoff-rigidity of
earths B-field) amd one based on the internal distribution of measured events.
Both are still in active development and might not be able to produce realistic
spectra for all observations. In that case a simple reference observation of one
of the RXTE background fields might be used instead. 
Both tools are further described on the instruments webpage: 
https://heasarc.gsfc.nasa.gov/docs/nicer/tools/nicer_bkg_est_tools.html

=== Issues to keep in mind: ===

Feel free to expand this list whenever you came across any new issues.

''' Optical loading: '''

When optical light is reflected onto the detectors, this can
significantly increase the low-energy background below 3keV. This is
most significant for solar angles below ~70 degrees. However, due to
its position on the ISS this can also happen at any solar angle when
light is reflected by other part of the ISS.

''' Polar Horns: '''

Unlike other observatories in LEO the ISS is on an orbit with a quite
high inclination of 51.6 degrees. This means the ISS passes through
the so called polar horn regions, where the outer Van Allen belt reach
LEO. These regions can feature an increased population of energetic
electrons, which can induce a strong background signal in NICER,
particular at higher energies. 
See this "science nugget" for more information: 
https://heasarc.gsfc.nasa.gov/docs/nicer/science_nuggets/20180809.html

''' Overly aggressive filtering: '''

It can happen, that e.g. optical loading leads to a filter parameter
exceeding the standard filter criteria for most of certain
observations. Such an observation might still be of interest, however.
In that case one might want to increase the allowed range e.g. for
"underonly_range" in "xtiprepare".

''' Large event files: '''

The downloaded eventfiles are generally compressed and are extracted to memory
before the data processing. This fails for large eventfiles of several GB.
In this case one has to unzip the file event_cl/niXXXXXXXXXX_0mpu7_ufa.evt.gz
before running xtiprepare.

''' Active FPMs: '''

NICER has 56 focal-plane modules (FPMs) of which 52 are usually active. For
different reasons, e.g. annealing, sometimes some FPMs are switched off. And
sometimes you have excluded certain FPMs because of their increased background
(especially FPM 14 & 34). In both cases you have to be careful to scale the ARF
or generate a new one. So one should check the event files how many detectors where
active during the observation. 
More info here: https://heasarc.gsfc.nasa.gov/docs/nicer/analysis_threads/arf-rmf/