'Apache Beam - FileIO. File output name

When I use this piece of code:

    stringRows.apply(TextIO.<String>write().withShardNameTemplate("")
            .to("gs://alpha-beta-gamma/one/two/processed/abc")
            .withNumShards(1)
            .withSuffix(".csv"));

the file produced will have a filename of abc.csv because of .withShardNameTemplate("") not abc-00000-of-00001.csv which I don't want

when I use this:

    stringRows.apply("sink",
            FileIO.<String>write()
                    .via(TextIO.sink())
                    .to("gs://alpha-beta-gamma/one/two/processed/")
                    .withNumShards(1)
                    .withPrefix("abc")
                    .withSuffix(".csv")
    );

produces a file called abc-00000-of-00001.csv So when using FileIO is there an equivalent of .withShardNameTemplate("") so that I don't get the shard references in the file name so I get abc.csv ?

Many Thanks, Andy

javaapache-beam-io


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source

Related Questions