Skip to content

Commit

Permalink
Add finetuning
Browse files Browse the repository at this point in the history
  • Loading branch information
pilotpirxie committed Oct 12, 2023
1 parent 259a2fe commit 754a12a
Show file tree
Hide file tree
Showing 9 changed files with 220 additions and 21 deletions.
Binary file modified .DS_Store
Binary file not shown.
147 changes: 146 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,146 @@
# ormGPT
# ormGPT

An ORM based on OpenAI that translates plain language into SQL queries and executes them on a database.

Currently supports database dialects: MySQL, PostgreSQL, and SQLite.

Supported languages: English, German, French, Spanish, Polish, Italian, Dutch, Portuguese, Ukrainian, Arabic, Chinese, Japanese, Korean, Turkish and many more.

```shell
ormgpt.query("give me post with id 1, all comments for this post and user information about author");
```

Generated query:
```sql
SELECT
p.id AS post_id,
p.title,
p.body,
c.id AS comment_id,
c.body AS comment_body,
u.username AS author_username,
u.email AS author_email
FROM
posts p
JOIN comments c ON p.id = c.post_id
JOIN users u ON u.id = p.user_id
WHERE
p.id = 1;
```

Response:
```js
[
{
post_id: 1,
title: 'Hello world!',
body: 'This is my first post!',
comment_id: 1,
comment_body: 'Hello world!',
author_username: 'test',
author_email: 'test@example.com'
}
]
```

![./preview.gif](./preview.gif)

## Installation

```shell
npm install ormgpt

# or

yarn add ormgpt

# or

pnpm add ormgpt
```

## Usage
Prepare a database schema file, for example `schema.sql`. This file will be used to generate queries.

```js
const client = await createConnection({
host: 'localhost',
port: 3306,
database: 'ormgpt',
user: 'root',
password: 'mysecretpassword',
});

const mysqlAdapter = new MysqlAdapter({
client
});

const ormgpt = new ormGPT({
apiKey: "OPENAI_API_KEY",
schemaFilePath: "./example/schema.sql",
dialect: "postgres",
dbEngineAdapter: mysqlAdapter,
});

await ormgpt.query(
"add new user with username 'test' and email 'test@example.com'",
);

const users = await ormgpt.query("get all users");
console.log(users);
```

### Adapters

MySQL
```js
const client = await createConnection({
host: 'localhost',
port: 3306,
database: 'ormgpt',
user: 'root',
password: 'mysecretpassword',
});

const mysqlAdapter = new MysqlAdapter({
client
});
```

Postgres
```js
const client = new Client({
host: 'localhost',
port: 5432,
database: 'ormgpt',
user: 'mysecretuser',
password: 'mysecretpassword',
});
client.connect();

const postgresAdapter = new PostgresAdapter({
client
});
```

SQLite
```js
const sqliteAdapter = new SqliteAdapter({
dbFilePath: "./example/db.sqlite",
});

const ormgpt = new ormGPT({
apiKey: process.env.OPENAI_API_KEY || "",
schemaFilePath: "./example/schema.sql",
dialect: "postgres",
dbEngineAdapter: sqliteAdapter,
});
```

### Why?

In the last two years, I found ORMs to be new "days since the last javascript framework" in the JavaScript ecosystem. And since AI is a hot buzzword
I decided to combine both and create an ORM that uses OpenAI to generate SQL queries. Please don't use this in production.

### License
MIT
Binary file modified example/db.sqlite
Binary file not shown.
7 changes: 6 additions & 1 deletion example/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,18 @@ import {ormGPT} from "../src";
);
console.log(postQuery2);

const postQuery3 = await ormgpt.getQuery(
"give me post with id 1, all comments for this post and user information about author",
);
console.log(postQuery3);

const commentQuery = await ormgpt.getQuery(
"give me all comments written between 2023-01-01 and 2023-12-01",
);
console.log(commentQuery);

const commentQuery2 = await ormgpt.getQuery(
"write a comment for post with id 1 with content 'Hello world!' as user with id 3",
"add a comment for post with id 1 with content 'Hello world!' as user with id 3",
);
console.log(commentQuery2);

Expand Down
4 changes: 2 additions & 2 deletions example/mysql.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,15 @@ import {MysqlAdapter} from "../src/MysqlAdapter";
password: 'mysecretpassword',
});

const postgresAdapter = new MysqlAdapter({
const mysqlAdapter = new MysqlAdapter({
client
});

const ormgpt = new ormGPT({
apiKey: process.env.OPENAI_API_KEY || "",
schemaFilePath: "./example/schema.sql",
dialect: "postgres",
dbEngineAdapter: postgresAdapter,
dbEngineAdapter: mysqlAdapter,
});

await ormgpt.query(
Expand Down
23 changes: 22 additions & 1 deletion example/sqlite.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,30 @@ import { SqliteAdapter } from "../src/SqliteAdapter";
});

await ormgpt.query(
"add new user with username 'test' and email 'test@example.com'",
"add new user with id 1, username 'test' and email 'test@example.com'",
);

const users = await ormgpt.query("get all users");
console.log(users);

await ormgpt.query(
"add new user with id 2, username 'test2' and email 'test2@example.com'"
);

await ormgpt.query(
"add new post with id 1, title 'Hello world!' and content 'This is my first post!' as user with id 1"
);

await ormgpt.query(
"add new post with id 2, title 'Hello world 2!' and content 'This is my second post!' as user with id 2"
);

await ormgpt.query(
"add comment to post with id 1 with comment content 'Hello world!', comment id 1 and add as user with id 2"
);

const postQuery3 = await ormgpt.query(
"give me post with id 1, all comments for this post and user information about author",
);
console.log(postQuery3);
})();
Binary file added preview.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
7 changes: 7 additions & 0 deletions src/ModelTuning.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
export type ModelTuning = {
temperature: number;
max_tokens: number;
top_p: number;
frequency_penalty: number;
presence_penalty: number;
}
53 changes: 37 additions & 16 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,39 +3,64 @@ import path from "path";
import {ErrorResponse} from "./ErrorResponse";
import {SuccessResponse} from "./SuccessResponse";
import {DatabaseEngineAdapter} from "./DatabaseEngineAdapter";
import {ModelTuning} from "./ModelTuning";

export class ormGPT {
private apiKey: string;
private apiUrl: string = "https://api.openai.com/v1/chat/completions";
private dbSchema: string;
private dialect: string;
private dbEngineAdapter?: DatabaseEngineAdapter;
private model: string = "gpt-3.5-turbo";
private modelOptions: ModelTuning = {
temperature: 1,
max_tokens: 256,
top_p: 1,
frequency_penalty: 0,
presence_penalty: 0,
}

constructor({
apiKey,
dialect,
schemaFilePath,
dbEngineAdapter,
}: {
apiKey,
dialect,
schemaFilePath,
dbEngineAdapter,
apiUrl,
model,
modelOptions
} : {
apiKey: string;
schemaFilePath: string;
dialect: "postgres" | "mysql" | "sqlite";
dbEngineAdapter?: DatabaseEngineAdapter;
apiUrl?: string;
model?: string;
modelOptions?: ModelTuning;
}) {
this.apiKey = apiKey;
this.dbSchema = fs.readFileSync(path.resolve(schemaFilePath), "utf-8");
this.dialect = dialect;
this.dbEngineAdapter = dbEngineAdapter;

if (apiUrl) {
this.apiUrl = apiUrl;
}
if (model) {
this.model = model;
}
if (modelOptions) {
this.modelOptions = modelOptions;
}
}

private async getResponse(request: string): Promise<string> {
const prompt = `
You are a SQL engine brain.
You are an SQL engine brain.
You are using ${this.dialect} dialect.
Having db schema as follow:
Having db schema as follows:
${this.dbSchema}
Write a query to fulfill the user request: ${request}
Write a query to fulfil the user request: ${request}
Don't write anything else than SQL query.
`;
Expand All @@ -47,18 +72,14 @@ export class ormGPT {
Authorization: `Bearer ${this.apiKey}`,
},
body: JSON.stringify({
model: "gpt-3.5-turbo",
model: this.model,
messages: [
{
role: "user",
content: prompt,
},
],
temperature: 1,
max_tokens: 256,
top_p: 1,
frequency_penalty: 0,
presence_penalty: 0,
...this.modelOptions,
}),
});

Expand All @@ -75,7 +96,7 @@ export class ormGPT {
try {
return await this.getResponse(request);
} catch (error) {
console.error("Error when executing query", request);
console.error("Error when generating query", request);
throw error;
}
}
Expand All @@ -86,7 +107,7 @@ export class ormGPT {
throw new Error("No dbEngineAdapter provided");
}

const query = await this.getResponse(request);
const query = await this.getQuery(request);
console.log("Executing query", query);
return this.dbEngineAdapter.executeQuery(query);
} catch (error) {
Expand Down

0 comments on commit 754a12a

Please sign in to comment.