Category "apache-flink"

Flink Python Datastream API Kafka Consumer

Im new to pyflink. Im tryig to write a python program to read data from kafka topic and prints data to stdout. I followed the link Flink Python Datastream API K

Flink Avro Error Expecting type to be a PojoTypeInfo

I'm trying to convert a HashMap<String, Object> to an Avro record. I get this runtime exception when I do a DataStream<AvroRecord> dsRpvSchema = fil

Table API's Over aggregation delays firing the trigger

Below is the code snippet: // Firstly the inputStream receives the first batch of 420 records; // Secondly the inputStream receives the second batch of

Is the description of the "Contiguity within looping patterns " in the document correct?

as the description in flink CEP document: Strict Contiguity: Expects all matching events to appear strictly one after the other, without any non-matching events

Flink "Encountered error while consuming partitions" + "Connection reset by peer"

I have a Flink streaming job running 24/7. Several times per day, I see it fail and restart with the following log messages: 10:02:08.524 [Flink Netty Server (0

Flink Job submission throws java.nio.file.NoSuchFileException while the file actually exists

I tried to submit a flink job that is already packaged in a JAR. Basically it consumes a kafka topic protected by SASL authentication, thus it requires a .jks f

Flink Table print connector not being called

I am using the Flink table API to pull data from a kinesis topic into a table. I want to periodically pull that data into a temporary table and run a custom sca

How to reference nested JSON within PyFlink SQL when JSON schema varies?

I have a stream of events I wish to process using PyFlink, where the events are taken from AWS EventBridge. The events in this stream share a number of common

Flink - InstanceAlreadyExistsException: while migrating to the KafkaSource

I am using flink with v1.13.2 . And I am trying to migrate FlinkKafkaConsumer to KafkaSource. While i am testing new KafkaSource, i am getting the following exc

JcaPEMKeyConverter is provided by BouncyCastle, an optional dependency. To use support for EC Keys you must explicitly add dependency to classpath

I have a simple Flink streaming app. It runs well in a cluster created by start-cluster.sh command. Now based on the Flink tutorial, I hope to deploy it in appl

JcaPEMKeyConverter is provided by BouncyCastle, an optional dependency. To use support for EC Keys you must explicitly add dependency to classpath

I have a simple Flink streaming app. It runs well in a cluster created by start-cluster.sh command. Now based on the Flink tutorial, I hope to deploy it in appl

flink cluster with zookeeper HA always shutdown: [RECEIVED SIGNAL 15: SIGTERM]

Environment: flink1.14.4 standalone application mode in kubernetes according to official steps: flink cluster: https://nightlies.apache.org/flink/flink-docs-rel

How to trigger a line of code in window based CEP

I have a stream of events and want to count the number of events in a specific period time to find out event loss. My code is something similar to the following

What's wrong with my Pyflink setup that Python UDFs throw py4j exceptions?

I'm playing with the flink python datastream tutorial from the documentation: https://ci.apache.org/projects/flink/flink-docs-master/docs/dev/python/datastream_

Can I direct Apache Beam late data into another output?

Is it possible to route late data into another output instead of dropping it? I see that Flink supports this - getting late data as a side output.

Apache Flink Stream Event Delay

I tried to develop the following code but it doesn't work. I would like to use apache Flink to delay the event that have the time (specified in the timestamp fi

Flink 1.13.2: NoResourceAvailableException

This is with Flink 1.13.2 running in Amazon's Kinesis Data Analytics Flink environment. This application is running on Kafka topics. When the topics had smaller

Deserialize Avro from kafka as SpecificRecord Failing. Expecting type to be a PojoTypeInfo

I am using Flink v1.11.2 and Avro v1.10.1. I am trying to deserialize an Avro record as a Specific record from a Kafka topic, but for some reason keep getting t

Transform Stream

I have a GenericRecord stream with value deserialised using Avro, schema has name and age. KafkaSource<GenericRecord> source = KafkaSource.<GenericRec

How to use ISO-8601 date in flink SQL?

Based on my research Flink SQL accepts "0000-01-01 00:00:00.000000000" as the timestamp format, but my timestamps in kafka are coming in "0000-01-01T00:00:00.00