How to create a single RDD[Genotype] object out of multiple VCF files?

[rezacsedu]

In Scala, I can read a single VCF file as follows:

val allGenotypes: RDD[Genotype] = sc.loadGenotypes(genotypeFile).rdd

However, I'm wondering if I can read multiple VCF files to create a single RDD[Genotype] object?

[heuermh]

Hello @rezacsedu!

You may either use globbed paths on load

val genotypes = sc.loadGenotypes("hdfs://data/sample*.vcf")
val rdd: RDD[Genotype] = genotypes.rdd

or union multiple GenotypeRDDs together

val genotypes0 = sc.loadGenotypes("sample0.vcf")
val genotypes1 = sc.loadGenotypes("sample1.vcf")
val union = genotypes0.union(genotypes1)
val rdd: RDD[Genotype] = union.rdd

[rezacsedu]

@heuermh thanks so much for prompt reply. Really appreciated!