Complex Types

[HaroonSaid]

Hi Tomas

Is there a way to create a table, using a model that has arrays, my Kafka topic looks like
The approach that I took, was to create a CREATE TYPE and CREATE TABLE using the cli.
However, it would be nice if I can do it thru the c# client

    public abstract class AbstractProducerClass
    {
        public string Id { get; set; }
        public string Key { get; set; }
    }
    public class EventCategory
    {
        public int Id { get; set; }
        public string Name { get; set; }
        public string Description { get; set; }
    }
    // The Topic that needs the table
    public class ElasticSearchEvent : AbstractProducerClass
    {
        public string Title { get; set; }
        public string Description { get; set; }
        public long EventId { get; set; }
        public IEnumerable<EventCategory> EventCategories { get; set; }
    }
}

Thanks

[HaroonSaid]

Thanks for the guidance.

[tomasfabian]

Hi @HaroonSaid,
C# version of Create type is not supported (implemented) in the current version.

Did you want to do something similar?

await restApiClient.CreateTableAsync<ElasticSearchEvent>(metadata);

Could you also provide the KSQL you used in the CLI mentioned above?

see also supported data types in selects

In selects try to use C# arrays instead of IEnumerable<T>

[HaroonSaid]

A KSQL fragment create fragment

CREATE TYPE EventCategories AS STRUCT<id INTEGER, name VARCHAR, description VARCHAR>;

    CREATE TABLE IF NOT EXISTS "enrichedevents" ( 
        key VARCHAR PRIMARY KEY,
        id VARCHAR,
        ...
        eventCategories ARRAY<EventCategories>)
    WITH (KAFKA_TOPIC='enrichedevents', PARTITIONS=1, VALUE_FORMAT='JSON');

[tomasfabian]

I will try to prioritize this requirement, at the moment you can execute any arbitrary string statements from C# in the following manner:

using Kafka.DotNet.ksqlDB.KSql.RestApi;
using Kafka.DotNet.ksqlDB.KSql.RestApi.Statements;

internal async Task ExecuteStatementAsync()
{
  var ksqlDbUrl = @"http:\\localhost:8088";

  var httpClientFactory = new HttpClientFactory(new Uri(ksqlDbUrl));

  IKSqlDbRestApiClient restApiClient = new KSqlDbRestApiClient(httpClientFactory);

  var statement = @"CREATE TYPE EventCategories AS STRUCT<id INTEGER, name VARCHAR, description VARCHAR>;";

  KSqlDbStatement ksqlDbStatement = new(statement);

  var httpResponseMessage = await restApiClient.ExecuteStatementAsync(ksqlDbStatement);
}

[tomasfabian]

hi @HaroonSaid,
could you please try out the above requested feature?

CreateTypeAsync

Install-Package Kafka.DotNet.ksqlDB -Version 1.6.0-rc.1

var  httpResponseMessage = await restApiClient.CreateTypeAsync<EventCategory>();

[tomasfabian]

I released v1.6.0