|
|
|
|
| BCFtools implements utilities for variant calling (in conjunction with
SAMtools) and manipulating VCF and BCF files. The program is intended
to replace the Perl-based tools from vcftools.
See INSTALL for building and installation instructions.
Please cite this paper when using BCFtools for your publications:
Twelve years of SAMtools and BCFtools
Petr Danecek, James K Bonfield, Jennifer Liddle, John Marshall, Valeriu Ohan, Martin O Pollard, Andrew Whitwham, Thomas Keane, Shane A McCarthy, Robert M Davies, Heng Li
GigaScience, Volume 10, Issue 2, February 2021, giab008, https://doi.org/10.1093/gigascience/giab008
@article{10.1093/gigascience/giab008,
author = {Danecek, Petr and Bonfield, James K and Liddle, Jennifer and Marshall, John and Ohan, Valeriu and Pollard, Martin O and Whitwham, Andrew and Keane, Thomas and McCarthy, Shane A and Davies, Robert M and Li, Heng},
title = "{Twelve years of SAMtools and BCFtools}",
journal = {GigaScience},
volume = {10},
number = {2},
year = {2021},
month = {02},
abstract = "{SAMtools and BCFtools are widely used programs for processing and analysing high-throughput sequencing data. They include tools for file format conversion and manipulation, sorting, querying, statistics, variant calling, and effect analysis amongst other methods.The first version appeared online 12 years ago and has been maintained and further developed ever since, with many new features and improvements added over the years. The SAMtools and BCFtools packages represent a unique collection of tools that have been used in numerous other software projects and countless genomic pipelines.Both SAMtools and BCFtools are freely available on GitHub under the permissive MIT licence, free for both non-commercial and commercial use. Both packages have been installed \\>1 million times via Bioconda. The source code and documentation are available from https://www.htslib.org.}",
issn = {2047-217X},
doi = {10.1093/gigascience/giab008},
url = {https://doi.org/10.1093/gigascience/giab008},
note = {giab008},
eprint = {https://academic.oup.com/gigascience/article-pdf/10/2/giab008/36332246/giab008.pdf},
}
|
|
|
|
| Written by Brian Bushnell
Last modified May 8, 2020
Contents:
This directory contains a collection of scripts written for calling variants and generating consensus genomes from SARS-CoV-2 ("Covid") Illumina data using BBTools. They were designed and optimized for several sets of libraries (from NextSeq and NovaSeq platforms) containing a mix of shotgun and amplicon data using various primers, and may need adjustment for your specific experimental design.
These scripts, and those in BBTools, should work fine in Bash on Linux, MacOS, or Windows 10. Other shells generally work fine.
Usage:
These scripts are just guidelines showing how I am processing Covid data.
If you want to use them without modification, follow these steps:
1) Install BBTools (you need the latest version - 38.85+ - due to some new features I added for Covid!).
a) If you don't have Java, install Java.
i) If you run java --version on the command line and it reports version 8+, you're fine.
ii) Otherwise, you can get it from https://openjdk.java.net/install/index.html
b) Download BBTools 38.85 or higher from https://sourceforge.net/projects/bbmap/
c) Unzip the archive: tar -xzf BBMap_38.85.tar.gz
d) Add it to the path: export PATH=/path/to/bbmap/:$PATH
i) Now you can run the shell scripts from wherever.
e) Samtools is not necessary, but recommended if you want to make the bam files for visualization.
2) Rename and interleave the files if they are paired, so libraries are in the format "prefix.fq.gz" with one file per library.
3) Copy the Covid reference to the directory.
4) Modify the template processCoronaWrapper.sh:
a) Pick a library to use for quality-score calibration if desired (line 16).
b) Add lines, or make a loop, so that processCorona.sh is called on all of the libraries (lines 20-21).
c) Delete lines 8-9 to let the script run.
5) Run processCoronaWrapper.sh.
Note:
If you need to perform dehosting (removal of human reads), I would suggest doing that first prior to any other data manipulations such as trimming or filtering to allow the resultant dehosted data to be as close to the raw data as possible.
You can remove human reads while maintaining fastq format and keeping pairing and read headers intact with a command like this (where reads.fq can be single-ended or paired/interleaved):
bbmap.sh ref=hg19.fa in=reads.fq outm=human.fq outu=nonhuman.fq bloom
...where hg19.fa would be a human reference. This will remove both reads in a pair if either of them map to human.
|
|
|
|
| -=- MUMmer3.x INSTALLATION README -=-
-- TO INSTALL --
Extract the tarfile by typing:
"tar -xvzf MUMmer3.x"
After extracting the tarfile into the desired installation directory,
change to the "MUMmer3.x" directory. Once in this directory, type:
"make check"
This command will check for the necessary utilities that the MUMmer package
depends on. If you see no error messages, the check was successful. Next, type:
"make install"
This command will build the appropriate scripts and binaries in the current
directory. Refer to the "README" file in this directory for further assistance,
or the "docs" directory for detailed information on the various utilities.
To make all of the scripts and executables accessible from different
directories, simply add the full MUMmer directory path to your system PATH, or
link the desired MUMmer programs to your favorite bin directory. Please note
that the 'make' command dynamically generates the MUMmer scripts with the
appropriate paths, therefore if the MUMmer directory is moved after the 'make'
command is issued, the scripts will fail to run. If the MUMmer executables
are needed in a directory other than the install directory, it is recommended
that the install directory be left untouched and its files linked to the
desired destination. An alternative would be to move the install directory and
reissue the 'make' command at the new location.
-- SYSTEM REQUIREMENTS --
MUMmer3.10 has been tested successfully on:
- Redhat Linux 6.2, 7.3 (Pentium 4)
- Compaq Tru64 UNIX 5.1 (alpha)
- SunOS UNIX 5.8 (sparc)
- Mac OS X 10.2.8 (PowerPC G4)
Sufficient memory and disk space are necessary, but required sizes vary
with input size. Be aware of your disk and memory usage, because insufficient
capacities will result in incorrect or missing output. Required resources
differ depending on the input size, but in general 512 MB of RAM and 1 GB of
disk space is sufficient for most applications.
It is possible to port the code to any system with a C++ compiler but
this has not been tested and will not be supported. In addition, you may need
to alter the Makefile to direct 'make' to your native compiler and other
system resources.
For Mac OSX, the Mac development kit must be downloaded and installed. This
kit will include 'gcc', 'ar', and 'make' which are necessary for building
MUMmer. MUMmer is not supported for any Mac operating system other than OSX.
-- SOFTWARE REQUIREMENTS --
The MUMmer3.x package requires the following to run successfully. In the
absence of one or more of these utilities, certain MUMmer programs may fail
to run correctly. Listed in parenthesis are the versions used to test the
MUMmer package. These versions, or subsequent versions should assure the proper
execution of the various MUMmer programs. These utilities must be accessible
via the system path:
- make (GNU make 3.79.1)
- perl (PERL 5.6.0)
- sh (GNU sh 1.14.7)
- csh (tcsh 6.10.00)
- g++ (GNU gcc 2.95.3)
- sed (GNU sed 3.02)
- awk (GNU awk 3.0.4)
- ar (GNU ar 2.9.5)
-- OPTIONAL UTILITIES --
To use the visualization tools included with MUMmer, it may be necessary
to download and install the following utilities:
- fig2dev (fig2dev 3.2.3)
- gnuplot (gnuplot 4.0)
- xfig (xfig 3.2)
-- UN-INSTALLATION --
In the "MUMmer3.x" base directory type:
"make uninstall"
This will remove all binary executables and runnable scripts from the
base and aux_bin directories. The original source files will remain unchanged,
thus reverting the package to its post-untar, pre-install state.
Please address questions and bug reports to:
Last update: May 12, 2005
|
|
|
|
| README.txt: an index to idlelib files and the IDLE menu.
IDLE is Python's Integrated Development and Learning
Environment. The user documentation is part of the Library Reference and
is available in IDLE by selecting Help => IDLE Help. This README documents
idlelib for IDLE developers and curious users.
IDLELIB FILES lists files alphabetically by category,
with a short description of each.
IDLE MENU show the menu tree, annotated with the module
or module object that implements the corresponding function.
This file is descriptive, not prescriptive, and may have errors
and omissions and lag behind changes in idlelib.
IDLELIB FILES
Implementation files not in IDLE MENU are marked (nim).
Deprecated files and objects are listed separately as the end.
Startup
-------
__init__.py # import, does nothing
__main__.py # -m, starts IDLE
idle.bat
idle.py
idle.pyw
Implementation
--------------
autocomplete.py # Complete attribute names or filenames.
autocomplete_w.py # Display completions.
autoexpand.py # Expand word with previous word in file.
browser.py # Create module browser window.
calltip_w.py # Display calltip.
calltips.py # Create calltip text.
codecontext.py # Show compound statement headers otherwise not visible.
colorizer.py # Colorize text (nim)
config.py # Load, fetch, and save configuration (nim).
configdialog.py # Display user configuration dialogs.
config_help.py # Specify help source in configdialog.
config_key.py # Change keybindings.
dynoption.py # Define mutable OptionMenu widget (nim).
debugobj.py # Define class used in stackviewer.
debugobj_r.py # Communicate objects between processes with rpc (nim).
debugger.py # Debug code run from shell or editor; show window.
debugger_r.py # Debug code run in remote process.
delegator.py # Define base class for delegators (nim).
editor.py # Define most of editor and utility functions.
filelist.py # Open files and manage list of open windows (nim).
grep.py # Find all occurrences of pattern in multiple files.
help.py # Display IDLE's html doc.
help_about.py # Display About IDLE dialog.
history.py # Get previous or next user input in shell (nim)
hyperparser.py # Parse code around a given index.
iomenu.py # Open, read, and write files
macosx.py # Help IDLE run on Macs (nim).
mainmenu.py # Define most of IDLE menu.
multicall.py # Wrap tk widget to allow multiple calls per event (nim).
outwin.py # Create window for grep output.
paragraph.py # Re-wrap multiline strings and comments.
parenmatch.py # Match fenceposts: (), [], and {}.
pathbrowser.py # Create path browser window.
percolator.py # Manage delegator stack (nim).
pyparse.py # Give information on code indentation
pyshell.py # Start IDLE, manage shell, complete editor window
query.py # Query user for information
redirector.py # Intercept widget subcommands (for percolator) (nim).
replace.py # Search and replace pattern in text.
rpc.py # Communicate between idle and user processes (nim).
rstrip.py # Strip trailing whitespace.
run.py # Manage user code execution subprocess.
runscript.py # Check and run user code.
scrolledlist.py # Define scrolledlist widget for IDLE (nim).
search.py # Search for pattern in text.
searchbase.py # Define base for search, replace, and grep dialogs.
searchengine.py # Define engine for all 3 search dialogs.
stackviewer.py # View stack after exception.
statusbar.py # Define status bar for windows (nim).
tabbedpages.py # Define tabbed pages widget (nim).
textview.py # Define read-only text widget (nim).
tree.py # Define tree widget, used in browsers (nim).
undo.py # Manage undo stack.
windows.py # Manage window list and define listed top level.
zoomheight.py # Zoom window to full height of screen.
Configuration
-------------
config-extensions.def # Defaults for extensions
config-highlight.def # Defaults for colorizing
config-keys.def # Defaults for key bindings
config-main.def # Defai;ts fpr font and geneal
Text
----
CREDITS.txt # not maintained, displayed by About IDLE
HISTORY.txt # NEWS up to July 2001
NEWS.txt # commits, displayed by About IDLE
README.txt # this file, displayed by About IDLE
TODO.txt # needs review
extend.txt # about writing extensions
help.html # copy of idle.html in docs, displayed by IDLE Help
Subdirectories
--------------
Icons # small image files
idle_test # files for human test and automated unit tests
Unused and Deprecated files and objects (nim)
---------------------------------------------
tooltip.py # unused
IDLE MENUS
Top level items and most submenu items are defined in mainmenu.
Extensions add submenu items when active. The names given are
found, quoted, in one of these modules, paired with a '<>'.
Each pseudoevent is bound to an event handler. Some event handlers
call another function that does the actual work. The annotations below
are intended to at least give the module where the actual work is done.
'eEW' = editor.EditorWindow
File
New File # eEW.new_callback
Open... # iomenu.open
Open Module # eEw.open_module
Recent Files
Class Browser # eEW.open_class_browser, browser.ClassBrowser
Path Browser # eEW.open_path_browser, pathbrowser
---
Save # iomenu.save
Save As... # iomenu.save_as
Save Copy As... # iomenu.save_a_copy
---
Print Window # iomenu.print_window
---
Close # eEW.close_event
Exit # flist.close_all_callback (bound in eEW)
Edit
Undo # undodelegator
Redo # undodelegator
--- # eEW.right_menu_event
Cut # eEW.cut
Copy # eEW.copy
Paste # eEW.past
Select All # eEW.select_all (+ see eEW.remove_selection)
--- # Next 5 items use searchengine; dialogs use searchbase
Find # eEW.find_event, search.SearchDialog.find
Find Again # eEW.find_again_event, sSD.find_again
Find Selection # eEW.find_selection_event, sSD.find_selection
Find in Files... # eEW.find_in_files_event, grep
Replace... # eEW.replace_event, replace.ReplaceDialog.replace
Go to Line # eEW.goto_line_event
Show Completions # autocomplete extension and autocompleteWidow (&HP)
Expand Word # autoexpand extension
Show call tip # Calltips extension and CalltipWindow (& Hyperparser)
Show surrounding parens # parenmatch (& Hyperparser)
Shell # pyshell
View Last Restart # pyshell.PyShell.view_restart_mark
Restart Shell # pyshell.PyShell.restart_shell
Interrupt Execution # pyshell.PyShell.cancel_callback
Debug (Shell only)
Go to File/Line
debugger # debugger, debugger_r, PyShell.toggle_debugger
Stack Viewer # stackviewer, PyShell.open_stack_viewer
Auto-open Stack Viewer # stackviewer
Format (Editor only)
Indent Region # eEW.indent_region_event
Dedent Region # eEW.dedent_region_event
Comment Out Reg. # eEW.comment_region_event
Uncomment Region # eEW.uncomment_region_event
Tabify Region # eEW.tabify_region_event
Untabify Region # eEW.untabify_region_event
Toggle Tabs # eEW.toggle_tabs_event
New Indent Width # eEW.change_indentwidth_event
Format Paragraph # paragraph extension
---
Strip tailing whitespace # rstrip extension
Run (Editor only)
Python Shell # pyshell
---
Check Module # runscript
Run Module # runscript
Options
Configure IDLE # eEW.config_dialog, configdialog
(tabs in the dialog)
Font tab # config-main.def
Highlight tab # query, config-highlight.def
Keys tab # query, config_key, config_keys.def
General tab # config_help, config-main.def
Extensions tab # config-extensions.def, corresponding .py
---
Code Context (ed)# codecontext extension
Window
Zoomheight # zoomheight extension
---
# windows
Help
About IDLE # eEW.about_dialog, help_about.AboutDialog
---
IDLE Help # eEW.help_dialog, helpshow_idlehelp
Python Doc # eEW.python_docs
Turtle Demo # eEW.open_turtle_demo
---
(right click)
Defined in editor, PyShelpyshellut
Cut
Copy
Paste
---
Go to file/line (shell and output only)
Set Breakpoint (editor only)
Clear Breakpoint (editor only)
Defined in debugger
Go to source line
Show stack frame
Center Insert # eEW.center_insert_event
CODE STYLE -- Generally PEP 8.
import
------
Put import at the top, unless there is a good reason otherwise.
PEP 8 says to group stdlib, 3rd-party dependencies, and package imports.
For idlelib, the groups are general stdlib, tkinter, and idlelib.
Sort modules within each group, except that tkinter.ttk follows tkinter.
Sort 'from idlelib import mod1' and 'from idlelib.mod2 import object'
together by module, ignoring within module objects.
Put 'import __main__' after other idlelib imports.
Imports only needed for testing are put not at the top but in an
htest function def or "if __name__ == '__main__'" clause.
Within module imports like "from idlelib.mod import class" may cause
circular imports to deadlock. Even without this, circular imports may
require at least one of the imports to be delayed until a function call.
|
|