Table of Contents

KEFCore: Entity Framework Core provider for Apache Kafka™

KEFCore is the Entity Framework Core provider for Apache Kafka™. Based on KNet client-side features it allows to use Apache Kafka™ as a distributed database and more: KNet client-side features are also compatible with any broker that implements the Kafka wire protocol — see Backend compatibility below.

Libraries and Tools

Core Templates Json Serialization Avro Serialization Protobuf Serialization
Core downloads Templates downloads Serialization downloads Serialization Avro downloads Serialization Protobuf downloads

Pipelines

CI_BUILD CI_RELEASE

Project disclaimer

KEFCore is a project curated by MASES Group and supported by the open-source community. Its primary scope is to support other MASES Group projects — both open-source and commercial — though it is freely available for any use. Dedicated community and commercial subscription plans are available. The repository and releases may contain bugs. The release cycle depends on critical issues discovered and/or enhancement requests from this or other dependent projects.

Looking for Entity Framework Core and Apache Kafka™ expertise? MASES Group can help you design, build, deploy, and manage Entity Framework Core and Apache Kafka™ applications. Find out more.

Scope of the project

KEFCore provides an Entity Framework Core provider for Apache Kafka™, enabling .NET applications to use Kafka topics as a data store through the standard EF Core programming model — DbContext, LINQ queries, and strongly-typed entities — with no Kafka-specific consumer or producer code.

The EF Core introduction page opens with this example:

public class BloggingContext : DbContext
{
    public DbSet<Blog> Blogs { get; set; }
    public DbSet<Post> Posts { get; set; }

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        optionsBuilder.UseSqlServer(
            @"Server=(localdb)\mssqllocaldb;Database=Blogging;Trusted_Connection=True;ConnectRetryCount=0");
    }
}

With KEFCore, replacing the SQL Server backend with an Apache Kafka™ cluster requires changing a single line:

optionsBuilder.UseKEFCore("my-application", "localhost:9092");

From that point on, standard EF Core code works unchanged against Kafka topics:

// Query
var blogs = await db.Blogs
    .Where(b => b.Rating > 3)
    .OrderBy(b => b.Url)
    .ToListAsync();

// Write
db.Blogs.Add(new Blog { Url = "http://sample.com" });
await db.SaveChangesAsync();

KEFCore is developed following the guidelines in the official EntityFrameworkCore repository and the Writing a provider documentation published by Microsoft.

Currently the project tries to support, at our best, the official supported Apache Kafka™ binary distribution:

KEFCore State KNet Apache Kafka™ .NET JVM™
2.6.+ Active 3.2.x 4.2.x .NET 8+ Java 17+
2.5. Deprecated 2.9.x 3.9.x .NET 8+ Java 11+

Backend compatibility

Entity Framework Core provider for Apache Kafka™ uses the official Apache Kafka™ Java client packages directly through KNet client-side features. This architecture has a direct impact on backend compatibility.

Client-side features — Producer, Consumer, Admin Client, Kafka Streams, KNet Streams SDK, KNet Connect SDK, KNetPS scriptable cmdlets — communicate with the broker exclusively through the Kafka wire protocol and are therefore compatible with any broker that implements it, not only Apache Kafka™ itself.

Examples of compatible brokers: Redpanda, Amazon MSK, Confluent Platform / Cloud, Aiven for Apache Kafka™, IBM Event Streams, WarpStream, AutoMQ, and others.

See Supported Backends for the full compatibility matrix covering all KNet feature areas.

Community and Contribution

If you find Entity Framework Core provider for Apache Kafka™ useful:

  • Leave a ⭐ on the repository
  • Open issues to report bugs 🐛 or request features
  • Submit Pull Requests to improve the project

This project adheres to the Contributor Covenant code of conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to coc_reporting@masesgroup.com.

Edit this page