Skip to content

uncommittedOffsets : Clarification and Alternatives #351

@vikas-goswami

Description

@vikas-goswami

Hi Team,

We are migrating to Javascript client from KafkaJS and using APIs under:
const { Kafka } = require("@confluentinc/kafka-javascript").KafkaJS;

We would like to get clarification on uncommittedOffsets support in the migration documentation present here:
Within the eachBatch callback, use of uncommittedOffsets is unsupported
Link to documentation

Is there any alternative provided to track and commit offsets with a similar fine-tuned control? I am attaching some example usages below.

Example Usage 1:

await resolveOffsets(batch.messages, resolveOffset)
await commitOffsetsIfNecessary(uncommittedOffsets())

Usage 2: Where we track and update uncommittedOffsets

const uncommittedOffsets = []
....
consumer.uncommittedOffsets = consumer.uncommittedOffsets.concat(uncommittedOffsets)
....
  if (consumer.uncommittedOffsets.length === 0) {
    return done(null, false)
  }
  consumer.commitOffsetsIfNecessary(consumer.uncommittedOffsets)
    .then(() => {
      consumer.uncommittedOffsets = []
      return done(null, true)
    })
    .catch(err => {
      return done(err, false)
    })
}

We do not want to use autocommit :true since, it does not provide the finer-grained control over when and how offsets are committed. So, please let us know if there's any alternative.

Thanks in advance.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions