kkonganti@11: # bettercallsal_db
kkonganti@11:
kkonganti@11: `bettercallsal_db` is an end-to-end automated workflow to generate and consolidate the required DB flat files based on [NCBI Pathogens Database for Salmonella](https://ftp.ncbi.nlm.nih.gov/pathogen/Results/Salmonella/). It first downloads the metadata based on the provided release identifier (Ex: `latest_snps` or `PDG000000002.2727`) and then creates a `mash sketch` based on the filtering strategy. It generates two types of sketches, one that prioritizes genome collection based on SNP clustering (`per_snp_cluster`) and the other just collects up to N number of genome accessions for each `computed_serotype` column from the metadata file (`per_computed_serotype`).
kkonganti@11:
kkonganti@11: The `bettercallsal_db` workflow should finish within an hour with stable internet connection.
kkonganti@11:
kkonganti@11: \
kkonganti@11:
kkonganti@11:
kkonganti@11: ## Workflow Usage
kkonganti@11:
kkonganti@11: ```bash
kkonganti@11: cpipes --pipeline bettercallsal_db [options]
kkonganti@11: ```
kkonganti@11:
kkonganti@11: \
kkonganti@11:
kkonganti@11:
kkonganti@11: Example: Run the `bettercallsal_db` pipeline and store output at `/data/Kranti_Konganti/bettercallsal_db/PDG000000002.2727`.
kkonganti@11:
kkonganti@11: ```bash
kkonganti@11: cpipes
kkonganti@11: --pipeline bettercallsal_db \
kkonganti@11: --pdg_release PDG000000002.2727 \
kkonganti@11: --output /data/Kranti_Konganti/bettercallsal_db/PDG000000002.2727
kkonganti@11: ```
kkonganti@11:
kkonganti@11: \
kkonganti@11:
kkonganti@11:
kkonganti@11: Now you can run the `bettercallsal` workflow with the created database by mentioning the root path to the database with `--bcs_root_dbdir` option.
kkonganti@11:
kkonganti@11: ```bash
kkonganti@11: cpipes
kkonganti@11: --pipeline bettercallsal \
kkonganti@11: --input /path/to/illumina/fastq/dir \
kkonganti@11: --output /path/to/output \
kkonganti@11: --bcs_root_dbdir /data/Kranti_Konganti/bettercallsal_db/PDG000000002.2727
kkonganti@11: ```
kkonganti@11:
kkonganti@11: \
kkonganti@11:
kkonganti@11:
kkonganti@11: ## Note
kkonganti@11:
kkonganti@11: Please note that the last step of the `bettercallsal_db` workflow named `SCAFFOLD_GENOMES` will spawn multiple processes and is not cached by **Nextflow**. This is an intentional setup for this specific stage of the workflow to speed up database creation and as such it is recommended that you run this workflow in a grid computing or similar cloud computing setting.
kkonganti@11:
kkonganti@11: \
kkonganti@11:
kkonganti@11:
kkonganti@11: ## `bettercallsal_db` CLI Help
kkonganti@11:
kkonganti@11: ```text
kkonganti@11: [Kranti_Konganti@my-unix-box ]$ cpipes --pipeline bettercallsal_db --help
kkonganti@11: N E X T F L O W ~ version 23.04.3
kkonganti@11: Launching `./bettercallsal/cpipes` [special_brenner] DSL2 - revision: 8da4e11078
kkonganti@11: ================================================================================
kkonganti@11: (o)
kkonganti@11: ___ _ __ _ _ __ ___ ___
kkonganti@11: / __|| '_ \ | || '_ \ / _ \/ __|
kkonganti@11: | (__ | |_) || || |_) || __/\__ \
kkonganti@11: \___|| .__/ |_|| .__/ \___||___/
kkonganti@11: | | | |
kkonganti@11: |_| |_|
kkonganti@11: --------------------------------------------------------------------------------
kkonganti@11: A collection of modular pipelines at CFSAN, FDA.
kkonganti@11: --------------------------------------------------------------------------------
kkonganti@11: Name : bettercallsal
kkonganti@11: Author : Kranti Konganti
kkonganti@11: Version : 0.6.1
kkonganti@11: Center : CFSAN, FDA.
kkonganti@11: ================================================================================
kkonganti@11:
kkonganti@11: Workflow : bettercallsal_db
kkonganti@11:
kkonganti@11: Author : Kranti Konganti
kkonganti@11:
kkonganti@11: Version : 0.6.1
kkonganti@11:
kkonganti@11:
kkonganti@11: Required :
kkonganti@11:
kkonganti@11: --output : Absolute path to directory where all the
kkonganti@11: pipeline outputs should be stored. Ex: --
kkonganti@11: output /path/to/output
kkonganti@11:
kkonganti@11: Other options :
kkonganti@11:
kkonganti@11: --wcomp_serocol : Column number (non 0-based index) of the
kkonganti@11: PDG metadata file by which the serotypes
kkonganti@11: are collected. Default: false
kkonganti@11:
kkonganti@11: --wcomp_complete_sero : Skip indexing serotypes when the serotype
kkonganti@11: name in the column number 49 (non 0-based)
kkonganti@11: of PDG metadata file consists a "-". For
kkonganti@11: example, if an accession has a serotype=
kkonganti@11: string as such in column number 49 (non 0-
kkonganti@11: based): "serotype=- 13:z4,z23:-" then, the
kkonganti@11: indexing of that accession is skipped.
kkonganti@11: Default: false
kkonganti@11:
kkonganti@11: --wcomp_not_null_serovar : Only index the computed_serotype column i.e
kkonganti@11: . column number 49 (non 0-based), if the
kkonganti@11: serovar column is not NULL. Default: false
kkonganti@11:
kkonganti@11: --wcomp_i : Force include this serovar. Ignores --
kkonganti@11: wcomp_complete_sero for only this serovar.
kkonganti@11: Mention multiple serovars separated by a
kkonganti@11: ! (Exclamation mark). Ex: --
kkonganti@11: wcomp_complete_sero I 4,[5],12:i:-!Agona
kkonganti@11: Default: false
kkonganti@11:
kkonganti@11: --wcomp_num : Number of genome accessions to be collected
kkonganti@11: per serotype. Default: false
kkonganti@11:
kkonganti@11: --wcomp_min_contig_size : Minimum contig size to consider a genome
kkonganti@11: for indexing. Default: false
kkonganti@11:
kkonganti@11: --wsnp_serocol : Column number (non 0-based index) of the
kkonganti@11: PDG metadata file by which the serotypes
kkonganti@11: are collected. Default: false
kkonganti@11:
kkonganti@11: --wsnp_complete_sero : Skip indexing serotypes when the serotype
kkonganti@11: name in the column number 49 (non 0-based)
kkonganti@11: of PDG metadata file consists a "-". For
kkonganti@11: example, if an accession has a serotype=
kkonganti@11: string as such in column number 49 (non 0-
kkonganti@11: based): "serotype=- 13:z4,z23:-" then, the
kkonganti@11: indexing of that accession is skipped.
kkonganti@11: Default: true
kkonganti@11:
kkonganti@11: --wsnp_not_null_serovar : Only index the computed_serotype column i.e
kkonganti@11: . column number 49 (non 0-based), if the
kkonganti@11: serovar column is not NULL. Default: false
kkonganti@11:
kkonganti@11: --wsnp_i : Force include this serovar. Ignores --
kkonganti@11: wsnp_complete_sero for only this serovar.
kkonganti@11: Mention multiple serovars separated by a
kkonganti@11: ! (Exclamation mark). Ex: --
kkonganti@11: wsnp_complete_sero I 4,[5],12:i:-!Agona
kkonganti@11: Default: 'I 4,[5],12:i
kkonganti@11:
kkonganti@11: --wsnp_num : Number of genome accessions to collect per
kkonganti@11: SNP cluster. Default: false
kkonganti@11:
kkonganti@11: --mashsketch_run : Run `mash screen` tool. Default: true
kkonganti@11:
kkonganti@11: --mashsketch_l : List input. Lines in each specify
kkonganti@11: paths to sequence files, one per line.
kkonganti@11: Default: true
kkonganti@11:
kkonganti@11: --mashsketch_I : ID field for sketch of reads (
kkonganti@11: instead of first sequence ID). Default:
kkonganti@11: false
kkonganti@11:
kkonganti@11: --mashsketch_C : Comment for a sketch of reads (
kkonganti@11: instead of first sequence comment). Default
kkonganti@11: : false
kkonganti@11:
kkonganti@11: --mashsketch_k : K-mer size. Hashes will be based on
kkonganti@11: strings of this many nucleotides.
kkonganti@11: Canonical nucleotides are used by default (
kkonganti@11: see Alphabet options below). (1-32) Default
kkonganti@11: : 21
kkonganti@11:
kkonganti@11: --mashsketch_s : Sketch size. Each sketch will have
kkonganti@11: at most this many non-redundant min-hashes
kkonganti@11: . Default: 1000
kkonganti@11:
kkonganti@11: --mashsketch_i : Sketch individual sequences, rather than
kkonganti@11: whole files, e.g. for multi-fastas of
kkonganti@11: single-chromosome genomes or pair-wise gene
kkonganti@11: comparisons. Default: false
kkonganti@11:
kkonganti@11: --mashsketch_S : Seed to provide to the hash
kkonganti@11: function. (0-4294967296) [42] Default:
kkonganti@11: false
kkonganti@11:
kkonganti@11: --mashsketch_w : Probability threshold for warning
kkonganti@11: about low k-mer size. (0-1) Default: false
kkonganti@11:
kkonganti@11: --mashsketch_r : Input is a read set. See Reads options
kkonganti@11: below. Incompatible with --mashsketch_i.
kkonganti@11: Default: false
kkonganti@11:
kkonganti@11: --mashsketch_b : Use a Bloom filter of this size (
kkonganti@11: raw bytes or with K/M/G/T) to filter out
kkonganti@11: unique k-mers. This is useful if exact
kkonganti@11: filtering with --mashsketch_m uses too much
kkonganti@11: memory. However, some unique k-mers may
kkonganti@11: pass erroneously, and copies cannot be
kkonganti@11: counted beyond 2. Implies --mashsketch_r.
kkonganti@11: Default: false
kkonganti@11:
kkonganti@11: --mashsketch_m : Minimum copies of each k-mer
kkonganti@11: required to pass noise filter for reads.
kkonganti@11: Implies --mashsketch_r. Default: false
kkonganti@11:
kkonganti@11: --mashsketch_c : Target coverage. Sketching will
kkonganti@11: conclude if this coverage is reached before
kkonganti@11: the end of the input file (estimated by
kkonganti@11: average k-mer multiplicity). Implies --
kkonganti@11: mashsketch_r. Default: false
kkonganti@11:
kkonganti@11: --mashsketch_g : Genome size (raw bases or with K/M/
kkonganti@11: G/T). If specified, will be used for p-
kkonganti@11: value calculation instead of an estimated
kkonganti@11: size from k-mer content. Implies --
kkonganti@11: mashsketch_r. Default: false
kkonganti@11:
kkonganti@11: --mashsketch_n : Preserve strand (by default, strand is
kkonganti@11: ignored by using canonical DNA k-mers,
kkonganti@11: which are alphabetical minima of forward-
kkonganti@11: reverse pairs). Implied if an alphabet is
kkonganti@11: specified with --mashsketch_a or --
kkonganti@11: mashsketch_z. Default: false
kkonganti@11:
kkonganti@11: --mashsketch_a : Use amino acid alphabet (A-Z, except BJOUXZ
kkonganti@11: ). Implies --mashsketch_n --mashsketch_k 9
kkonganti@11: . Default: false
kkonganti@11:
kkonganti@11: --mashsketch_z : Alphabet to base hashes on (case
kkonganti@11: ignored by default; see --mashsketch_Z). K-
kkonganti@11: mers with other characters will be ignored
kkonganti@11: . Implies --mashsketch_n. Default: false
kkonganti@11:
kkonganti@11: --mashsketch_Z : Preserve case in k-mers and alphabet (case
kkonganti@11: is ignored by default). Sequence letters
kkonganti@11: whose case is not in the current alphabet
kkonganti@11: will be skipped when sketching. Default:
kkonganti@11: false
kkonganti@11:
kkonganti@11: Help options :
kkonganti@11:
kkonganti@11: --help : Display this message.
kkonganti@11:
kkonganti@11: ```