Class KafkaMessageEnumerator<K,​V>

  • Type Parameters:
    K - : type for Kafka message key, refer to ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG;
    V - : type for Kafka message value, refer to ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG;
    All Implemented Interfaces:
    java.lang.AutoCloseable, Enumerator<java.lang.Object[]>

    public class KafkaMessageEnumerator<K,​V>
    extends java.lang.Object
    implements Enumerator<java.lang.Object[]>
    Enumerator to read data from Consumer, and converted into SQL rows with KafkaRowConverter.
    • Field Summary

      Fields 
      Modifier and Type Field Description
      private java.util.LinkedList<org.apache.kafka.clients.consumer.ConsumerRecord<K,​V>> bufferedRecords  
      private java.util.concurrent.atomic.AtomicBoolean cancelFlag  
      (package private) org.apache.kafka.clients.consumer.Consumer consumer  
      private org.apache.kafka.clients.consumer.ConsumerRecord<K,​V> curRecord  
      (package private) KafkaRowConverter<K,​V> rowConverter  
    • Constructor Summary

      Constructors 
      Constructor Description
      KafkaMessageEnumerator​(org.apache.kafka.clients.consumer.Consumer consumer, KafkaRowConverter<K,​V> rowConverter, java.util.concurrent.atomic.AtomicBoolean cancelFlag)  
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()
      Closes this enumerable and releases resources.
      java.lang.Object[] current()
      It returns an Array of Object, with each element represents a field of row.
      boolean moveNext()
      Advances the enumerator to the next element of the collection.
      private void pullRecords()  
      void reset()
      Sets the enumerator to its initial position, which is before the first element in the collection.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • consumer

        final org.apache.kafka.clients.consumer.Consumer consumer
      • cancelFlag

        private final java.util.concurrent.atomic.AtomicBoolean cancelFlag
      • bufferedRecords

        private final java.util.LinkedList<org.apache.kafka.clients.consumer.ConsumerRecord<K,​V>> bufferedRecords
      • curRecord

        private org.apache.kafka.clients.consumer.ConsumerRecord<K,​V> curRecord
    • Constructor Detail

      • KafkaMessageEnumerator

        KafkaMessageEnumerator​(org.apache.kafka.clients.consumer.Consumer consumer,
                               KafkaRowConverter<K,​V> rowConverter,
                               java.util.concurrent.atomic.AtomicBoolean cancelFlag)
    • Method Detail

      • current

        public java.lang.Object[] current()
        It returns an Array of Object, with each element represents a field of row.
        Specified by:
        current in interface Enumerator<K>
        Returns:
        Current element
      • moveNext

        public boolean moveNext()
        Description copied from interface: Enumerator
        Advances the enumerator to the next element of the collection.

        After an enumerator is created or after the reset method is called, an enumerator is positioned before the first element of the collection, and the first call to the moveNext method moves the enumerator over the first element of the collection.

        If moveNext passes the end of the collection, the enumerator is positioned after the last element in the collection and moveNext returns false. When the enumerator is at this position, subsequent calls to moveNext also return false until #reset is called.

        An enumerator remains valid as long as the collection remains unchanged. If changes are made to the collection, such as adding, modifying, or deleting elements, the enumerator is irrecoverably invalidated. The next call to moveNext or Enumerator.reset() may, at the discretion of the implementation, throw a ConcurrentModificationException.

        Specified by:
        moveNext in interface Enumerator<K>
        Returns:
        true if the enumerator was successfully advanced to the next element; false if the enumerator has passed the end of the collection
      • pullRecords

        private void pullRecords()
      • reset

        public void reset()
        Description copied from interface: Enumerator
        Sets the enumerator to its initial position, which is before the first element in the collection.

        An enumerator remains valid as long as the collection remains unchanged. If changes are made to the collection, such as adding, modifying, or deleting elements, the enumerator is irrecoverably invalidated. The next call to Enumerator.moveNext() or reset may, at the discretion of the implementation, throw a ConcurrentModificationException.

        This method is optional; it may throw UnsupportedOperationException.

        Notes to Implementers

        All calls to Reset must result in the same state for the enumerator. The preferred implementation is to move the enumerator to the beginning of the collection, before the first element. This invalidates the enumerator if the collection has been modified since the enumerator was created, which is consistent with Enumerator.moveNext() and Enumerator.current().

        Specified by:
        reset in interface Enumerator<K>
      • close

        public void close()
        Description copied from interface: Enumerator
        Closes this enumerable and releases resources.

        This method is idempotent. Calling it multiple times has the same effect as calling it once.

        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface Enumerator<K>