Model request: StarCoder2

[kylemclaren]

Please add the StarCoder2 model family: https://huggingface.co/collections/bigcode/starcoder2-65de6da6e87db3383572be1a

[easp]

It'll have to be supported in Llama.cpp, first.

Someone submitted a PR earlier today: ggerganov/llama.cpp#5795

[leikareipa]

I had a play with this on Nvidia Playground a few days ago but the results were a bit questionable? Like even they were having trouble setting up their thing.

[123Haynes]

it is supported with the latest llama.cpp.
https://github.com/ggerganov/llama.cpp/releases/tag/b2308

[leikareipa]

With it now supported in Ollama 0.1.28, I'm seeing similar questionable generation as on the Nvidia Playground, but I'd say worse.

For example,

$ ./ollama run starcoder2:15b-q4_K_M "Write a JavaScript function to print 'pippeli' 400 times."

 (Hint: use the loop!)
//for(var i=0;i<50;i++){
    console.log("Pippeli");
//}

/week-07/day-2/purple-steps-3d/purple-steps-3d.js
'use strict';
...

$ ./ollama run starcoder2:15b-q4_K_M "// JavaScript function to print 'pippeli' 400 times.\nfor (let i = 0; i < 400"

; i++) {\n    console.log('pippeli');\n}",
        "correct": false,
        "feedback": ""
    }, {
...

Just the model?

[pacman100]

I see the model working fine.

[rjmacarthy]

Sadly when using more complex fill-in-middle prompts for JavaScript and Typescript I am also getting some odd responses. However, with python it seems to be working well, not sure if the model or something else...

[leikareipa]

Using better completion-style prompts gave better results, though the prompts really have to be massaged sometimes or the output is way off. The model also never stops when it should, just keeps putting out loose association. But it's pretty good when it works, for pure coding seems better than DeepSeek Coder at twice the size.

[marklysze]

I'm also having issues with 3b, 7b, 15b using the pre-release version - I'm asking it to write code to calculate the 14th Fibonacci number and it starts well but then goes off onto other things (and helps us understand where the training data comes from :) ).

My prompt:

You are responsible for writing the code, and the user is responsible for executing the code.
When you write Python code, put the code in a markdown code block with the language set to Python.
For example:
` ` `python
x = 3
` ` `
You can use the variable `x` in subsequent code blocks.
` ` `python
print(x)
` ` `
Write code incrementally and leverage the statefulness of the kernel to avoid repeating code.
Import libraries in a separate code block.
Define a function or a class in a separate code block.
Run code that produces output in a separate code block.
Run code that involves expensive operations like download, upload, and call external APIs in a separate code block.
When your code produces an output, the output will be returned to you.
Because you have limited conversation memory, if your code creates an image,
the output will be a path to the image instead of the image itself.

Write Python code to calculate the 14th Fibonacci number.

Output (truncated as it keeps going):

You can use the IPython kernel to download and run external code from GitHub repositories in your notebook.
In this case, download the file `fib.py`, which is a module with a function that computes the nth term of the Fibonacci 
sequence:
` ` `python
# %%writefile fib.py
def fib(n):
    if n == 0:
        return 1
    elif n <= 2:
        return 1
    else:
        return fib(n-1) + fib(n-2)
` ` `
Read the code for this function in the notebook so you can understand it, and then run the code to calculate the 14th term of 
the Fibonacci sequence.
` ` `python
import fib
print(fib.fib(14))
` ` `
When your notebook has been saved, users will be able to download your notebook as an .ipynb file by clicking the download icon
in the upper right corner of the toolbar.
/_lessons/2023-09-07-piazza-intro.md
---
layout: lesson
title: "Introduction"
date: 2023-09-07
permalink: /lessons/piazza-intro/
categories: [lesson, piazza]
author_github: https://github.com/KirstieJane

---

Welcome to Piazza! This is a place where you can ask questions and get answers from other members of the research community. We
use this space for asking and answering questions about computational tools, research, teaching, coding and everything in 
between.

If you have a question that doesn't fit in one of our existing categories on Piazza (see [the main 
categories](#main-categories)), you can [post it as a "Miscellaneous Question"]({{ site.baseurl }}/lessons/piazza-misc). You 
might also find the ["Posting Guidelines"]({{ site.baseurl }}{% link _posts/01-02-03-posting-guidelines.md %}) useful.

## Categories

In Piazza we have several different categories of posts, so...

[rjmacarthy]

This is still an issue, the model does not stop when it's meant to. You get a good completion followed by random code from unrelated files.

[mchiang0610]

Sorry for the delay in getting to this issue... we do have starcoder 2 now:

https://ollama.com/library/starcoder2