KEFCore: Entity Framework Core provider for Apache Kafka™

KEFCore is the Entity Framework Core provider for Apache Kafka™. Based on KNet it allows to use Apache Kafka™ as a distributed database and more.

Libraries and Tools

Core	Templates	Json Serialization	Avro Serialization	Protobuf Serialization

Pipelines

Project disclaimer

KEFCore is a project, curated by MASES Group, can be supported by the open-source community.

Its primary scope is to support other, public or internal, MASES Group projects: open-source community and commercial entities can use it for their needs and support this project, moreover there are dedicated community and commercial subscription plans.

The repository code and releases may contain bugs, the release cycle depends from critical discovered issues and/or enhancement requested from this or other projects.

Looking for the help of Entity Framework Core and Apache Kafka™ experts? MASES Group can help you design, build, deploy, and manage Entity Framework Core and Apache Kafka™ applications.

Scope of the project

This project aims to create a provider able to save data to, and read data from, an Apache Kafka™ cluster using the paradigm behind Entity Framework.

Have you ever reached the Entity Framework Core introduction page? The first example proposed is:

using System.Collections.Generic;
using Microsoft.EntityFrameworkCore;

namespace Intro;

public class BloggingContext : DbContext
{
    public DbSet<Blog> Blogs { get; set; }
    public DbSet<Post> Posts { get; set; }

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        optionsBuilder.UseSqlServer(
            @"Server=(localdb)\mssqllocaldb;Database=Blogging;Trusted_Connection=True;ConnectRetryCount=0");
    }
}

public class Blog
{
    public int BlogId { get; set; }
    public string Url { get; set; }
    public int Rating { get; set; }
    public List<Post> Posts { get; set; }
}

public class Post
{
    public int PostId { get; set; }
    public string Title { get; set; }
    public string Content { get; set; }

    public int BlogId { get; set; }
    public Blog Blog { get; set; }
}

which uses a SQL Server instance to store and retrieve the data.

With Entity Framework Core provider for Apache Kafka™ any user can replace a database (the available providers always use a database) with an Apache Kafka™ cluster. So the following query:

using (var db = new BloggingContext())
{
    var blogs = await db.Blogs
        .Where(b => b.Rating > 3)
        .OrderBy(b => b.Url)
        .ToListAsync();
}

or the following operation:

using (var db = new BloggingContext())
{
    var blog = new Blog { Url = "http://sample.com" };
    db.Blogs.Add(blog);
    await db.SaveChangesAsync();
}

can be done on a set of topics in an Apache Kafka™ cluster.

The project is based on available information within the official EntityFrameworkCore repository, many classes was copied from there as reported in the official documentation within the Microsoft website at https://docs.microsoft.com/ef/core/providers/writing-a-provider.

Currently the project tries to support, at our best, the official supported Apache Kafka™ binary distribution:

KEFCore	State	KNet	Apache Kafka™	.NET	JVM™
2.6.+	Active	3.1.x	4.1.x
	2.5.	Deprecated	2.9.x	3.9.x

Community and Contribution

Do you like the project?

Request your free community subscription.

Do you want to help us?

put a ⭐ on this project
open issues to request features or report bugs 🐛
improves the project with Pull Requests