Revision as of 16:39, 6 September 2013

Utah Genome Project

Variant Calling Pipeline 0.0.2

Sept. 2013

Data Source

Data sets used for the variant calling pipeline come from the Broad GSA (GATK) group as the 'GATK resource bundle 2.5' version 2.5

wget -r ftp://gsapubftp-anonymous@ftp.broadinstitute.org/bundle/2.5/b37

Reference Genome (GRCh37):

human_g1K_v37.fasta

High confidence SNVs

 1000G_omni2.5.b37.vcf
 hapmap_3.3.b37.vcf

dbSNP

 dbsnp_137.b37.vcf.gz

High confidence indels

 Mills_and_1000G_gold_standard.indels.b37.vcf.gz

Sequencing

This pipeline is designed for 100 bp Illumina HiSeq PE exome or WGS sequence data with Sanger/Illumina 1.9 quality encoding.

Validate File Integrity with md5sum

An md5sum signature should be provided for each FastQ file by the sequencing center. After the file has been downloaded, locally check the md5sum to be sure that no data corruption occurred during the file transfer.

md5sum file.fastq > file_local.md5
diff file_local.md5 file_provided.md5

If the md5sum signature differs from that provided for the file:

Check to be sure you have the correct file.
Check if the md5sum was calculated on that compressed or uncompressed file by the provider and be sure to do the same with the local copy.
Try the download again.
Contact the sequence provider.

FastQ File Analyses

fastqc 9958X1_130327_700179R_0404_BD1YNUACXX_7_1.txt

From the fastqc_data.txt file we check the following values:

Encoding (must be Sanger / Illumina 1.9)
Total Sequences (Need to develop a acceptable range)
Filtered Sequences (Should be 0 or at least very low)
Sequence length (must be ~ 100 bp)
%GC (should be 45 < x < 55)
Total Duplicate Percentage (This value may not be valuable - an acceptable range has not been determined).

Alignment

Index the reference sequence if this has not already been done

bwa index -a bwtsw human_g1K_v37.fasta

The 'bwa aln' program will find the reference coordinates of the input reads (independent of their mate-pair). The following parameters are those used by the 1KG project for aligning Illumina data.

bwa aln -q 15 human_g1K_v37.fasta file.fastq > lane1_r1.sai # One lane of reads first in pair
bwa aln -q 15 human_g1K_v37.fasta file.fastq > lane1_r2.sai # One lane of reads second in pair

The 'bwa sampe'. For paired-end reads, the maximum insert size is taken to be 3 times the expected insert size.

bwa sampe -P human_g1K_v37.fasta lane1_r1.sai lane1_r2.sai lane1_r1.fastq lane1_r1.fastq | samtools view -bt human_g1K_v37.fasta.fai -o lane1 -

This will switch to bwa mem soon.

bwa mem human_g1K_v37.fasta lane1_r1.fq lane2_r2.fq | samtools view -bt human_g1K_v37.fasta.fai -o lane1 -

Bwa Command Line Options:

-n NUM max #diff (int) or missing prob under 0.02 err rate (float) [0.04]
-o INT maximum number or fraction of gap opens [1]
-e INT maximum number of gap extensions, -1 for disabling long gaps [-1]
-i INT do not put an indel within INT bp towards the ends [5]
-d INT maximum occurrences for extending a long deletion [10]
-l INT seed length [32]
-k INT maximum differences in the seed [2]
-m INT maximum entries in the queue [2000000]
-t INT number of threads [1]
-M INT mismatch penalty [3]
-O INT gap open penalty [11]
-E INT gap extension penalty [4]
-R INT stop searching when there are >INT equally best hits [30]
-q INT quality threshold for read trimming down to 35bp [0]
-c input sequences are in the color space
-L log-scaled gap penalty for long deletions
-N non-iterative mode: search for all n-difference hits (slooow)
-f FILE file to write output to instead of stdout

The AddOrReplaceReadGroups tool will add the correct readgroup ID to the BAM files

nohup java -d64 -Xmx8g -jar picard-tools/AddOrReplaceReadGroups.jar \
   INPUT=lane1.bam                                                  \
   OUTPUT=lane1_rg.bam                                              \
   CREATE_INDEX=TRUE                                                \
   VALIDATION_STRINGENCY=LENIENT                                    \
   SORT_ORDER=coordinate                                            \
   RGID=9958X1                                                      \
   RGLB=9958X1                                                      \
   RGPL=illumina                                                    \
   RGPU=1                                                           \
   TMP_DIR=/big/drive                                               \
   2> 9958X1_AddOrReplaceReadGroups.error

BAM File Analyses

Alignment BAM files are improved in various ways to help increase the quality and speed of subsequent variant calling steps.

Sort BAM

java -jar -Xmx3g picard-tools/SortSam.jar             \
   INPUT=/output/filename.b37_1kg.recal.bam            \
   OUTPUT=/output/filename.b37_1kg.recal.sorted.bam    \
   CREATE_INDEX=TRUE                                  \
   SORT_ORDER=coordinate                              \
   VALIDATION_STRINGENCY=LENIENT                      \
   TMP_DIR=/big/drive                                 \
   2> SortSam.error

Merge lane level BAMs to individual

java -jar -Xmx4g picard-tools/MergeSamFiles.jar \
    INPUT=lane1_.bam                            \
    INPUT=alignment.bam                         \                                                                                                 
    OUTPUT=merged.bam                           \
    CREATE_INDEX=TRUE                           \
    ASSUME_SORTED=true                          \
    USE_THREADING=true                          \
    SORT_ORDER=coordinate                       \
    VALIDATION_STRINGENCY=LENIENT               \
    TMP_DIR=/big/drive                          \
    2> MergeSameFiles.error

Mark Duplicates

Remove PCR/Optical duplicate reads

java $jvm_args -jar MarkDuplicates.jar \
   INPUT=lane1.markdup.bam             \
   OUTPUT=lane1_markdup                \
   ASSUME_SORTED=TRUE                  \
   METRICS_FILE=lane1_dup_metrics.txt  \
   VALIDATION_STRINGENCY=LENIENT       \
   TMP_DIR=/big/drive

GATK Local Realignment of Indels

java -Xmx10g -jar /usr/local/GenomeAnalysisTK-2.4-7-g5e89f01//GenomeAnalysisTK.jar \
   -I all_bams.list                             		                    \
   -T RealignerTargetCreator                    		                    \
   -R human_g1K_v37.fasta                               		            \
   -o lane_1.realign.intervals                    				    \
   -known Mills_and_1000G_gold_standard.indels.b37.vcf                             \
   -known 1000G_phase1.indels.b37.vcf.gz                                           \
   -nt 24                                                                          \
   >  RealignerTargetCreator                                                       \
   2> RealignerTargetCreator.error

java -Xmx10g -jar /usr/local/GenomeAnalysisTK-2.4-7-g5e89f01/GenomeAnalysisTK.jar \
   -T IndelRealigner                                                              \
   -R fasta/hs_ref_GRCh37.p10_ALL_chr_hs37d5.fa                                   \
   -I lane1_markdup.bam                                                           \
   -targetIntervals realign.intervals                                             \
   -o 9958X_Realigned.bam                                                         \
   -known Mills_and_1000G_gold_standard.indels.b37.vcf                            \
   -known 1000G_phase1.indels.b37.vcf.gz                                          \
   >  IndelRealigner.out                                                          \
   2> IndelRealigner.error

Base Quality Score Recalibration

CountCovariates and TableRecalibration are no longer options in GATK (http://gatkforums.broadinstitute.org/discussion/1248/countcovariates).
Use BaseRecalibrator for both of the below step now (http://www.broadinstitute.org/gatk/gatkdocs/org_broadinstitute_sting_gatk_walkers_bqsr_BaseRecalibrator.html)

java [jvm_args] -jar GenomeAnalysisTK.jar                                 \
     -T CountCovariates                                                   \
     -R $reference_fasta                                                  \
     -I $realigned_bam_file                                                \
     -recalFile recal_data.csv                                            \
     -knownSites $known_sites_file(s)                                      \
     -l INFO                                                              \
     -L '1;2;3;4;5;6;7;8;9;10;11;12;13;14;15;16;17;18;19;20;21;22;X;Y;MT' \
     -cov ReadGroupCovariate                                              \
     -cov QualityScoreCovariate                                           \
     -cov CycleCovariate                                                  \
     -cov DinucCovariate                                                  \

java [jvm_args] -jar GenomeAnalysisTK.jar \ 
     -T TableRecalibration                \
     -R $reference_fasta                  \
     -recalFile recal_data.csv            \
     -I $realigned_bam_file                \
     -o $recalibrated_bam_file             \
     -l INFO                              \
     -compress 0                          \
     -disable_bam_indexing                \

Known sites for recalibration are from dbSNP135:

ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/reference/phase2_mapping_resources/ALL.wgs.dbsnp.build135.snps.sites.vcf.gz

Sort and index recalibrated alignment (~5h)

java -jar -Xmx3g /tools/picard-tools-1.32/SortSam.jar \
   INPUT=/output/filename.b37_1kg.recal.bam            \
   OUTPUT=/output/filename.b37_1kg.recal.sorted.bam    \
   SORT_ORDER=coordinate                              \
   VALIDATION_STRINGENCY=LENIENT                      \
   TMP_DIR=/big/drive

java -jar -Xmx3g /tools/picard-tools-1.32/BuildBamIndex.jar \
   INPUT=/output/filename.b37_1kg.recal.sorted.bam           \
   OUTPUT=/output/filename.b37_1kg.recal.sorted.bam.bai      \
   VALIDATION_STRINGENCY=LENIENT \ TMP_DIR=/big/drive

Calculate covariates after realignment and recalibration

java -jar -Xmx2g GenomeAnalysisTK.jar                      \
   -l INFO                                                 \
   -T CountCovariates                                      \
   -U ALLOW_UNINDEXED_BAM                                  \
   -R /resources//hg19/indices/b37_1kg.fa                  \
   --DBSNP /resources//hg19/dbsnp/dbsnp_129_b37_b37_1kg.rod \
   -I /output/filename.b37_1kg.recal.sorted.bam             \
   -cov ReadGroupcovariate                                 \
   -cov QualityScoreCovariate                              \
   -cov CycleCovariate                                     \
   -cov DinucCovariate                                     \
   -recalFile /output/filename.b37_1kg.recal.covariate_table.csv

Analyze covariates before and after

java -jar -Xmx4g AnalyzeCovariates.jar                             \
   -l INFO                                                         \
   -resources /resources//hg19/indices/b37_1kg.fa                  \
   --recal_file /output/filename.b37_1kg.matefixed.covariate_table.csv \
   -outputDir /output/filename.b37_1kg.recal.stats_before/          \
   -Rscript ${rscript}
   -ignoreQ 5

java -jar -Xmx4g AnalyzeCovariates.jar                          \
   -l INFO                                                      \
   -resources /resources//hg19/indices/b37_1kg.fa               \
   --recal_file /output/filename.b37_1kg.recal.covariate_table.csv \
   -outputDir /output/filename.b37_1kg.recal.stats_after/        \
   -Rscript ${rscript}                                          \
   -ignoreQ 5

Generate the SAM MD Tag

The MD tag in SAM files provides a string representation of mismatching positions. The CIGAR string used in SAM files does not provide full detail of mismatch positions. The samtools calmd command is used to generate MD tags as well as which fix the NM tags (Edit distance to the reference) and introduce the BQ tags (Base Alignment Quality) which can be used during variant calling.

samtools calmd -Erb recalibrated_file.bam $reference.fasta > $bq_file.bam

Strip Extraneous BAM Tags

Run-level BAMs have extraneous tags (OQ, XM, XG, XO) stripped from them, to reduce total file size by around 30%.

GATK ReduceReads ???

Merge BAMS

Merge all BAMS for a given individual

java $jvm_args -jar MergeSamFiles.jar INPUT=$tag_stripped_bam_file(s) OUTPUT=$library_level_bam VALIDATION_STRINGENCY=LENIENT

Merge All BAMS for the Project

This happens above... Picard MergeSamFiles is used to Merge all the SAM files for a given project.

java $jvm_args -jar MergeSamFiles.jar INPUT=$markdup_bam_file(s) OUTPUT=$platform_level_bam VALIDATION_STRINGENCY=LENIENT

BAM Quality Control

Picard Tools
- BamIndexStats
- CalculateHsMetrics ??
- CleanSam
- CollectAlignmentSummaryMetrics
- CollectGcBiasMetrics
- CollectInsertSizeMetrics
- CollectMultipleMetrics
- CollectTargetedPcrMetrics
- EstimateLibraryComplexity
- FixMateInformation
- MarkDuplicates
- MeanQualityByCycle
- QualityScoreDistribution
- ValidateSamFile
GATK QC
[1]
[2]

CollectAlignmentSummaryMetrics

java -jar -Xmx4g /tools/picard-tools-1.32/CollectAlignmentSummaryMetrics.jar \
   I=/output/filename.b37_1kg.sorted.bam                                      \
   O=/output/filename.b37_1kg.AlignmentSummaryMetrics                         \
   R=/resources//hg19/indices/b37_1kg.fa                                     \
   VALIDATION_STRINGENCY=LENIENT                                             \
   TMP_DIR=/big/drive

CollectGcBiasMetrics

java -jar /tools/picard-tools-1.32/CollectGcBiasMetrics.jar \
   R=/resources//hg19/indices/b37_1kg.fa                    \
   I=/output/filename.b37_1kg.sorted.bam                     \
   O=/output/filename.b37_1kg.GcBiasMetrics                  \
   CHART=/output/filename.b37_1kg.GcBiasMetrics.pdf          \
   VALIDATION_STRINGENCY=LENIENT                            \
   TMP_DIR=/big/drive

CollectInsertSizeMetrics

java -jar /tools/picard-tools-1.32/CollectInsertSizeMetrics.jar \
   I=/output/filename.b37_1kg.sorted.bam                         \
   O=/output/filename.b37_1kg.CollectInsertSizeMetrics           \
   H=/output/filename.b37_1kg.CollectInsertSizeMetrics.pdf       \
   VALIDATION_STRINGENCY=LENIENT                                \
   TMP_DIR=/big/drive

MeanQualityByCycle

java -jar /tools/picard-tools-1.32/MeanQualityByCycle.jar \
   I=/output/filename.b37_1kg.sorted.bam                   \
   O=/output/filename.b37_1kg.MeanQualityByCycle           \
   CHART=/output/filename.b37_1kg.MeanQualityByCycle.pdf   \
   VALIDATION_STRINGENCY=LENIENT                          \
   TMP_DIR=/big/drive

QualityScoreDistribution

java  -jar  /tools/picard-tools-1.32/QualityScoreDistribution.jar       \
   I=/output/filename.b37_1kg.sorted.bam                                 \
   O=/output/filename.b37_1kg.[wiki:QualityScoreDistribution]            \
   CHART=/output/filename.b37_1kg.[wiki:QualityScoreDistribution].pdf    \
   VALIDATION_STRINGENCY=LENIENT                                        \
   TMP_DIR=/big/drive

BamIndexStats

java -jar /tools/picard-tools-1.32/BamIndexStats.jar \
   INPUT=/output/filename.b37_1kg.sorted.bam          \
   VALIDATION_STRINGENCY=LENIENT                     \
   TMP_DIR=/big/drive

CalculateHsMetricsWholeGenome

java -jar -Xmx3g /tools/picard-tools-1.32/CalculateHsMetricsWholeGenome.jar \
   INPUT=/output/filename.b37_1kg.sorted.bam                                 \
   OUTPUT=/output/filename.b37_1kg.HsMetrics                                 \
   BAIT_INTERVALS=/resources//hg19/intervals/GoNL.interval_list             \
   TARGET_INTERVALS=/resources//hg19/intervals/GoNL.interval_list           \
   VALIDATION_STRINGENCY=LENIENT                                            \
   TMP_DIR=/big/drive

Split BAMS by Chromosome

BAMs are split into chromosomes BAMs. These files move on to variant calling.

Compress BAMS with ReduceRead

Variant Calling

UnifiedGenotyper

java -jar GenomeAnalysisTK.jar \
  -R resources/Homo_sapiens_assembly18.fasta \
  -T UnifiedGenotyper \
  -I sample1.bam [-I sample2.bam ...] \
  --dbsnp dbSNP.vcf \
  -o snps.raw.vcf \
  -stand_call_conf [50.0] \
  -stand_emit_conf 10.0 \
  -dcov [50 for 4x, 200 for >30x WGS or Whole exome] \
  [-L targets.interval_list]

VariantRecalibrator

java -Xmx4g -jar GenomeAnalysisTK.jar \
  -T VariantRecalibrator \
  -R reference/human_g1k_v37.fasta \
  -input NA12878.HiSeq.WGS.bwa.cleaned.raw.subset.b37.vcf \
  -resource:hapmap,known=false,training=true,truth=true,prior=15.0 hapmap_3.3.b37.sites.vcf \
  -resource:omni,known=false,training=true,truth=false,prior=12.0 1000G_omni2.5.b37.sites.vcf \
  -resource:dbsnp,known=true,training=false,truth=false,prior=6.0 dbsnp_135.b37.vcf \
  -an QD -an HaplotypeScore -an MQRankSum -an ReadPosRankSum -an FS -an MQ -an InbreedingCoeff \
  -mode SNP \
  -recalFile path/to/output.recal \
  -tranchesFile path/to/output.tranches \
  -rscriptFile path/to/output.plots.R

ApplyRecalibration

java -Xmx3g -jar GenomeAnalysisTK.jar \
  -T ApplyRecalibration \
  -R reference/human_g1k_v37.fasta \
  -input NA12878.HiSeq.WGS.bwa.cleaned.raw.subset.b37.vcf \
  --ts_filter_level 99.0 \
  -tranchesFile path/to/output.tranches \
  -recalFile path/to/output.recal \
  -mode SNP \
  -o path/to/output.recalibrated.filtered.vcf

Difference between revisions of "UGP Variant Pipeline 0.0.2"

Revision as of 16:39, 6 September 2013

Contents

Utah Genome Project

Variant Calling Pipeline 0.0.2

Sept. 2013

Data Source

Sequencing

Validate File Integrity with md5sum

FastQ File Analyses

Alignment

BAM File Analyses

Sort BAM

Merge lane level BAMs to individual

Mark Duplicates

GATK Local Realignment of Indels

Base Quality Score Recalibration

Generate the SAM MD Tag

Strip Extraneous BAM Tags

Merge BAMS

Merge All BAMS for the Project

BAM Quality Control

Split BAMS by Chromosome

Compress BAMS with ReduceRead

Variant Calling

UnifiedGenotyper

VariantRecalibrator

ApplyRecalibration

Variant File Analyses

Family Based Analyses

IBD/SGS

Linkage

Population Based Analyses

Phasing

FST

Phenotype Based Analyses

GWAS

VAAST

Navigation menu

Search