usage: __main__.py [-h] [--json_prefix p] [--gt_dir DIR]
[--use_gt_masks_as_all_masks] [--dilate_endpoints NB_PASS]
[--remove_invalid] [--save_wpc_separately] [--compute_ic]
[--unique] [--remove_wpc_belonging_to_another_bundle]
[--no_empty] [--indent INDENT] [--sort_keys]
[--no_bbox_check] [--reference REFERENCE]
[-v [{DEBUG,INFO,WARNING,ERROR}]] [-f]
in_tractogram gt_config out_dir
Scores input tractogram overall and bundlewise, based on reference 'ground
truth' bundles. This script will segment a tractogram into many bundles using
a list of criteria for each (given in a config file), and compare the
segmented bundles with the reference bundles. The output metrics are the ones
suggested in the Tractometer publication [1]: dice score (also named F1),
overlap, overreach, true/false positives.
This script is used, for instance, to score the ISMRM Tractography Challenge
tractograms, with the new segmentation criteria suggested in [2].
See also
--------
- scil_tractogram_segment_with_bundleseg
- scil_tractogram_segment_connections_from_labels
- scil_bundle_score_many_bundles_one_tractogram: You may use this script
to score again the segmented bundles.
Outputs
-------
The output folder contains:
- results.json: The full tractometry report.
- processing_stats.json: Contains information on the segmentation of
bundles (ex: the number of WPC per criteria).
- segmented_VB/*_VS.trk.
- segmented_IB/*_*_IC.trk (if args.compute_ic)
- segmented_WPC/*_wpc.trk (if args.save_wpc_separately)
- IS.trk OR NC.trk (if args.compute_ic)
By default, if a streamline fits in many bundles, it will be included in every
one. If you want to assign each streamline to at most one bundle, use the
`--unique` flag.
Config file
-----------
The config file needs to be a json containing a dict of the ground-truth
bundles as keys. The value for each bundle is itself a dictionnary with:
Mandatory:
- endpoints OR [head AND tail]: filename for the endpoints ROI.
If 'enpoints' is used, we will automatically separate the mask into two
ROIs, acting as head and tail. Quality check is strongly recommended.
Optional:
Concerning metrics:
- gt_mask: expected result. OL and OR metrics will be computed from this.*
Concerning inclusion criteria (other streamlines will be WPC):
- all_mask: ROI serving as "all" criteria: to be included in the bundle,
ALL points of a streamline must be inside the mask.*
- any_mask: ROI serving as "any" criteria: streamlines
must touch that mask in at least one point ("any" point) to be included
in the bundle.
- angle: angle criteria. Streamlines containing loops and sharp turns above
given angle will be rejected from the bundle.
- length: maximum and minimum lengths per bundle.
- length_x / length_x_abs: maximum and minimum total distance in the x
direction (i.e. first coordinate).**
- length_y / length_y_abs: maximum and minimum total distance in the y
direction (i.e. second coordinate).**
- length_z / length_z_abs: maximum and minimum total distance in the z
direction (i.e. third coordinate).**
* Files must be .tck, .trk, .nii or .nii.gz. If it is a tractogram, a mask will
be created. If it is a nifti file, it will be considered to be a mask.
** With absolute values: looping will contribute to the total distance instead
of cancelling it.
Example config file:
{
"Ground_truth_bundle_0": {
"gt_mask": "PATH/bundle0.nii.gz",
"angle": 300,
"length": [140, 150],
"endpoints": "PATH/file1.nii.gz"
}
}
---------------------------------------------------------------------------------
References:
[1] Côté, M.-A., et al. (2013). Tractometer: Towards Validation of Tractography
Pipelines, Medical Image Analysis, 17(7), 844-857.
[2] Renauld, E., et al. (2023) Validate your white matter tractography
algorithms with a reappraised ISMRM 2015 Tractography Challenge scoring system,
Scientific Reports, 13:2347 (2023). Online Enhanced PDF, or Download PDF.
---------------------------------------------------------------------------------
Tractometry
-----------
Global connectivity metrics:
- Computed by default:
- VS: valid streamlines, belonging to a bundle (i.e. respecting all the
criteria for that bundle; endpoints, limit_mask, gt_mask.).
- IS: invalid streamlines. All other streamlines. IS = IC + NC.
- Optional:
- WPC: wrong path connections, streamlines connecting correct ROIs but not
respecting the other criteria for that bundle. Such streamlines always
exist but they are only saved separately if specified in the options.
Else, they are merged back with the IS.
By definition. WPC are only computed if "limits masks" are provided.
- IC: invalid connections, streamlines joining an incorrect combination of
ROIs. Use carefully, quality depends on the quality of your ROIs and no
analysis is done on the shape of the streamlines.
- NC: no connections. Invalid streamlines minus invalid connections.
- Fidelity metrics:
- OL: Overlap. Percentage of ground truth voxels containing streamline(s)
for a given bundle.
- OR: Overreach. Amount of voxels containing streamline(s) when they
shouldn't, for a given bundle. We compute two versions :
OR_pct_vs = divided by the total number of voxel covered by the bundle.
(percentage of the voxels touched by VS).
Values range between 0 and 100%. Values are not defined when we
recovered no streamline for a bundle, but we set the OR_pct_vs to 0
in that case.
OR_pct_gt = divided by the total size of the ground truth bundle mask.
Values could be higher than 100%.
- f1 score: which is the same as the Dice score.
positional arguments:
in_tractogram Input tractogram to score
gt_config .json dict configured as specified above.
out_dir Output directory for the resulting segmented bundles.
options:
-h, --help show this help message and exit
--json_prefix p Prefix of the two output json files. Ex: 'study_x_'.Files will be saved inside out_dir.
Suffixes will be 'processing_stats.json' and 'results.json'.
--no_empty Do not write file if there is no streamline.
--no_bbox_check Activate to ignore validity of the bounding box during loading / saving of
tractograms (ignores the presence of invalid streamlines).
--reference REFERENCE
Reference anatomy for tck/vtk/fib/dpy file
support (.nii or .nii.gz).
-v [{DEBUG,INFO,WARNING,ERROR}]
Produces verbose output depending on the provided level.
Default level is warning, default when using -v is info.
-f Force overwriting of the output files.
Additions to gt_config:
--gt_dir DIR Root path of the ground truth files listed in the gt_config.
If not set, filenames in the config file are considered
as absolute paths.
--use_gt_masks_as_all_masks
If set, the gt_config's 'gt_mask' will also be used as
'all_mask' for each bundle. Note that this means the
OR will necessarily be 0.
Preprocessing:
--dilate_endpoints NB_PASS
Dilate endpoint masks n-times. Default: 0.
--remove_invalid Remove invalid streamlines before scoring.
Tractometry choices:
--save_wpc_separately
If set, streamlines rejected from VC based on the config
file criteria will be saved separately from IS (and IC)
in one file *_wpc.tck per bundle.
--compute_ic If set, IS are split into NC + IC, where IC are computed as one bundle per
pair of ROI not belonging to a true connection, named
*_*_IC.tck.
--unique If set, streamlines are assigned to the first bundle they fit in and not to all.
--remove_wpc_belonging_to_another_bundle
If set, WPC actually belonging to any VB (in the
case of overlapping ROIs) will be removed
from the WPC classification.
Json options:
--indent INDENT Indent for json pretty print.
--sort_keys Sort keys in output json.
2.2.2