-
-
Notifications
You must be signed in to change notification settings - Fork 604
OSv Release 0.11
Tzach Livyatan edited this page Jul 31, 2014
·
2 revisions
We are pleased to announce the release of OSv 0.11 Alpha Release
Some of the noteworthy additions are:
We are continuously extending REST API coverage of OSv. Latest additions are:
- Network API: get routes information
- Hardware API: get processor numbers and flags
- File system API: df - report filesystem disk space usage
- OS API: update the OS command line
- Environment Variables API: unset an environment variable
Introducing a new optional Lua CLI module Lua CLI can run on OSv, when building the image with make -image=cli, or externally, using REST API to interact with the OSv server. Lua CLI have low footprint, and does not relay on the JVM. More on the Lua CLI here
The original memory allocation algorithm for large objects was first-fit linear search. The new allocator achieves better computational complexity, which is not worse than logarithmic as long as there is appropriately large amount of free contiguous memory. Small unallocated memory blocks are kept, depending on their size, in one of 16 lists with constant time allocation and deallocation. Free areas too large to fit in any of these lists are stored in a tree that guarantees logarithmic complexity.
now support sections in user data. Each section can refer to a particular module/application.
experimental, use at your own risk
collect entropy from interrupts, 1 bit from the following sources:
- interrupt vector number
- instruction pointer
- cycles counter and provides RDRAND as a source of entropy for CSPRNG
A new testing tool, which allows testing EC2 metadata query locally
Some of the latest improvements
- A new route cache mitigate TCP/IP scalability issues on SMP by remove locking (rwlock and mutex). Improves Tomcat benchmark by 10% for 4 vCPU use case.
- maximize LSO frames size, Maximizing the LSO packet size we reduce the CPU usage both in the guest, and in HV. improves the netperf TCP_STREAM test performance up to 260%, depend on packet size.
- Add fast path in ARP lookups, caching IPv4 address and MAC address
- Decrease lock contention within the TCP accept case
- Images of the new release are available at osv.io/downloads and using capstan search
- EC2 community AMI are available here
Amnon Heiman (127):
Http Server: remove unused config object
Http Serve: Clean up the matcher class
Http Server: Add mime types for ico and txt
Http Server: Os Version API
Http Server: Change the OS version stub to use uname
Http Server: Move handlers in the api directory
Http Server: Json formatter
Http Server: Change the OS version to work with the json formatter
Http Server: switch to boost_program_options-mt
Http Server: Add json path support
Http Server: Support Json objects
Http Server: Expend the formatter
Http Server: Routes can be configure with a json path
Http Server: auto code generation script
Http Server: Makefile support code generation
Http Server: Initilize the route from the json path definition
Http Server: Add stub for shutdown
Http Server: Implement the os api
Http Server: Code generation script error
Http Server: Code Cleanup in routes and matchrules
Http Server: Code clean up and astyle handlers
Http Server: Rename parser to transformer
Http server: file interaction handlers complete support for file_transformer
Http Server: Search a header in the request by its name
Http Server: add get_url to the request
Http Server: fix a bug in the matcher
Http Server: allows empty paramter in a path parameter
Http Server: add redirect on missing slash
Http Server: when requesting for a directory adds index.html to the path
Http Server: Add a file transformer
Http Server: Simplify the routes API with the url object
Http Server: Add file mapping
Http Server: Add file mapping to main
Http Server: missing a class in a method decleration
Http Server: Fix a bug in the initilization of the a bufferbuilder
Http Server: Remove copy constructor and assignment from json base
Http Server: Fix a few error with commas in Json parsing
Http Server: Adding copy constructor to the generated code
Http Server: Add query parameters to the request structure
Http Server: Fix a comment and format in handler.hh
Http Server: Add not found exception to handler
Http Server: support query parameters
Http Server: routes dispatching need to support GET/POST
Http Server: Add the java directory to the include list
Http Server: Add the jvm implementation
Http Server: Add the jvm init to main
Http Server: Add stubs for the java_api
Http Server: Add PUT and DELETE operation
Http Server: Support PUT and DELETE http command
Http Server: use typedef in connection.hh
Http Server: Add message handling exception to connection
Http Server: Add a helper read string function to connection
Http Server: Add multi-part field to the request
Http Server: Add multi-part parser to connection object
Http Server: Add method to check if this is a multi part message
Http Server: Read multi-part messages
Http Server: add bad parameter exception
Http Server: Adding the file API implementation
Http Server: Add file init to main.cc
Http Server: Change the Swagger GUI to / instead of /ui
Http Server: Use const for operation names
Http Server: Change file.cc to use constants over string
Http Server: Makefile should depend on the code generation script
Http Server: Override the default error message
Http Server: Add enum support to code generation
Http server: Additional file support
Http Server: Testing to support additional os version
Http Server testing: Adding file API testing
Http Server: Move exception to their own file
Http Server: File upload handled by the handler
Http Server: file api - file upload
Http Server: Testing the file upload API
Http server: Add stub for debugger buffer
Http Server: Makefile to depend on h files
Move the swagger api submodule to mgmt
Http Server: Clean up in the lambda expression handlers
Http Server: handler to support mandatory parameters
Http Server: add mandatory parameters to the path
Http server: Add a parameter not found exception
Http Server: Add mandatory parameters support to the routes
Http server: Adding path holder
Http server: code generation to support mandatory parameters
Http Server: Code generation declare path holder
Http Server: Initilialize the path_holder
Http Server: file.cc to use new API
Http Server: running astyle on jvm.cc
Http Server: jvm and os API implementation to use new API
Http Server: Modify the parameter order in path_holder
Http Server: set a default type in path holder to be json
Http Server: Move the parameters object to the request
Swagger API: Adding the env.json
Swagger API: Adding the env.json to the api-docs
Http Server: Adding the env API implementation
Http Server: Call env::init from main
Http Server: Add tests for environment variable
OSv Init: http client
OSv Init: the osvinit class
OSv Init: the main
OSv Init: A basic configuration file
Http Server: A singleton class for starting the httpserver
OSv Init: Makefile
Http server: Make file create a stub library
OSv Init: add the module.py
Http Server: rename httpserver.so to libhttpserver.so
OSvInit: add directory of the yaml-cpp library
Http Server: The global server header file
Http Server: Code generation, add template function to copy and set structure.
Http Server: Auto code generation determine class order
Http Server: Code generation removing tracing printouts
Http Server: Make file git pull should pull from the root directory
Http Server: Split Testing into multiple files
vfs mount: change char * to const char*
Http Server Testing: Adding test for thread API
Remove the Route gathering logic outside of lsroute
Http Server: Add routes API
Adding libtools.so library
Http Server: Write a text to a file
Http Server: code generation improve error handling
Http Server: Implicitly call json formatting for json handlers
build.mk pass arch to the module compilation
Adding libtools.so library
Http Server: clean up the implementation using the updated API
Drivers get_name method should be const
libtools should be added to the image
Move the get interface logic outside of ifconfig into libtools.lib
Http Server: Add an API to unset environment variable
unset environment varaible change the swagger definition
Asias He (11):
tcp: Increase TCBHASHSIZE from 512 to 4096
tcp: Remove the function tcp_twrecycleable
net: Decrease lock contention within the TCP accept case
net: Add fast path in ARP lookups
syscall: Drop #if 0'd sendfile code
net: Get rid of unnecessary recursive locking
net: kill lltable_drain
net: kill arp_ifinit2
net: Remove dead code in if.cc
net: Remove ifa_ref()/ifa_free(), which are atomic(9), from ip_output()
net: Do not leak ifa in ip_output
Avi Kivity (48):
httpserver: build with optimizations
httpserver: add build dependencies
vfs: move poll_link to file.h
vfs: convert file::f_poll_list to be a boost::intrusive::list
vfs: remove unneeded extern "C" from osv/file.h
epoll: use correct key
epoll: properly lock epoll file
epoll: handle races between epoll_wait() and epoll_ctl(EPOLL_CTL_DEL)
epoll: use epoll_file in epoll_ptr
tests: add epoll tests for same file/differnt fd
Merge tag 'epoll-fixes-v2' of https://github.com/avikivity/osv
test/misc-malloc: fix data corruption in worker thread
rcu hashtable
tests: add test for rcu_hashtable
net: use rcu_hashtable in net channel classifier
sched: fix allocation in interruptible wait_until
Revert "net: use rcu_hashtable in net channel classifier"
virtio: fix missed notification
build: teach osv-version.sh not to pick up random tags
tests: fix tst-dlfcn
tests: reduce rcu hashtable test running time
tests: run tests with --unsafe-cache
net: avoid more tcp_close() in tcp_do_segment()
libc: import if_nametoindex() from musl
Improve complexity of the large allocator
build: don't regenerate makefiles constantly
build: ask before nuking old build/release/
Revert "mempool: use new page range allocator"
Revert "build: ask before nuking old build/release/"
rcu_hashtable: fix bad erase()
Revert "Revert "net: use rcu_hashtable in net channel classifier""
bsd: avoid #define malloc() when possible
memory: allow using debug allocator in parallel with standard allocator
memory: debug allocator helper
Merge branch dbg-alloc
build: remove .SECONDARY
build: fix build/release link generation
net: kill inpgroups
net: simplify inpcb allocation
net: remove unused fields from struct inpcb
net: inpcb: replace bzero() with C++ initialization
net: remove tcp offload support
net: convert inpcbinfo::ipi_lock to a mutex
net: tcp: remove dead code
net: tcp: fix up indentations
Revert "build: remove .SECONDARY"
rcu hashtable: add documenation
rcu hashtable: fix documentation
Calle Wilund (8):
Remote REST API for trace point introspection and control
Add REST Api call for trace dump extraction.
java: Add chained exception to ContextIsolator:runjar
java: Fix JVM stack protection issues
mmu: file-mapped vma flag
libc: Fix pthread_getattr_np()
Add "modules" to symbol search path to get httpserver etc symbols
httpserver: Support POST requests with application/x-www-form-urlencoded
Claudio Fontana (3):
core/elf: validate NOTE segment length
aarch64: implement processor::ticks()
drivers: pl011: do not convert \n to \r\n
Dmitry Fleytman (1):
release-ec2: introduce AWS region/zone/placement group parameters
Elazar Leibovich (3):
x64: Document how boot16.S get loader size
x64: Replace magic constant with variable for ELF entry point offset
x64: explain where %rbp and %rbx are set
Glauber Costa (29):
arch: factor out parse_cmdline
inspect the current command line
commands: add method to save a command line to disk
create an exception for code 500
os: add an api to modify the command line
libc: regex functions
command line: fix issues with commited code
cpuid: add string to cpuid parser
processor: provide a string representation of all processor features
move cpuid.hh to processor.hh header
httpserver: harware probing API
xen blkfront: Don't for quiesce on absent flush
xen blkfront: software barrier support
pthread: implement pthread yield
libc: implement passwd handling functions
pthread: implement pthread_rwlock functions
user: implement some more user functions
user: implement getgrgid
user: nitpick for getlogin_r
build: export a meaningful arch-string to be used for OSv
use the canonical arch string for uname, instead of hard coding it
elf: initial implementation of getauxval
path lookup: allow null pointer for name argument
fs: implement lutimes
tests: return a failure code if tst-utimes fails
tst: increment tst-utimes with lutimes functionality
libc: implement glibc's glob_pattern_p
time functions: implement utime
httpserver: document /os/cmdline
Gleb Natapov (2):
vfs: increase vnode refcount under vnode_lock lock in dentry_alloc()
bsd: change zfs rwlock implementation to use OSv's rwlock instead of mutex
Jaspal Singh Dhillon (6):
add handler for /os/dmesg and add dmesg as a command
rest_api: expose set/get hostname api
vfs: implement sendfile()
tests: add tests for sendfile()
vfs: fix errno in sendfile()
tests: fix tst-sendfile.cc
Masami Ichikawa (1):
scripts: Support both python2 and python3 for api.py
Nadav Har'El (26):
mgmt: compile osvinit after httpserver
mgmt/httpserver: Add more include directories
REST API: Add /os/thread request
Remove bsd/include/pthread_np.h
Httpserver: add to "make clean"
readdir(): set missing d_reclen field
libosv.so: Add missing weak symbols
Fix thread::thread_clock() on a running thread
Update mgmt.git head
libc: snprintf for "infinite" lengths
Add __longjmp_chk()
command line: don't wait for disowned threads
loader: always run httpserver, and other stuff from /init
module system: add httpserver module unless explicitly removed
console: Fix bare linefeeds on early output on Xen
Another early-boot linefeed fix
Add missing backtrace line
tests: Test thread_local support
bsd: Change again recently-changed rwlock API
Re-add ".SECONDARY" to build.mk
Route Cache patch #1
Route Cache patch #2
Clean up tests/misc-mutex.cc
revert "make image=", add new "make modules="
Add rwlock read-lock benchmark to tests/misc-mutex
Fix empty-path case of futimesat()
Or Cohen (21):
Add ncurses module
Add libedit module
Add lua module
Add cli module code
Build code for cli module
Some documents created while working on the CLI
cli: Fix usage of httpserver module
cli: Use a proper terminal definition
httpserver: Fix /os/dmesg output format
cli: Refactoring command structure
cli: Add "hello" boilerplate command example
cli: Multiple modifications for API utility
cli: cat: use API to retrieve file content
cli: Add date command
cli: Add df command
cli: Add free command
cli: Add mkdir command
cli: Add rm command
cli: Add dmesg command
Lua: Install modules with LuaRocks
Lua: Add lua-stdlib to available Lua modules
Paweł Dziepak (14):
mempool: limit max_object_size to page_size / 4
mempool: replace MALLOC_ALIGNMENT with alignof(max_align_t)
mempool: set default alignment to min(size, alignof(max_align_t))
mempool: don't increase size of arbitrary aligned objects
mempool: make sure that alignof(max_align_t) >= 2 * sizeof(size_t)
include: add ilog2()
include: add count_trailing_zeros() functions
mempool: remove unused function debug_memory_pool()
mempool: introduce new page range allocator
mempool: use new page range allocator
mempool: fix page_range_allocator::_bitmap resize
Revert "Revert "mempool: use new page range allocator""
loader: get offset of list and set hooks in page_range
memory: don't allocate huge page range if it is not big enough
Pekka Enberg (33):
Rename miniweb to httpserver
Http Server: Fix small formatting glitch
Http Server: Clean up <api/os.hh>
Http Server: Fix include path in Makefile
Http Server: Show code generation during build
api-doc: update to latest
Merge httpserver from mgmt.git into osv.git
httpserver: Fix Swagger UI submodule
Merge pull request #1 from tgrabiec/httpserver
modules/httpserver: Fix path for 'make check'
mgmt: update to latest
scripts: GCE image uploading
mgmt: update to latest
mgmt: update to latest
Merge branch 'lua-cli'
Move osvinit into osv.git
modules/osvinit: Fix module configuration
modules/osvinit: Fix module path
mgmt: update to latest
Rename osvinit to cloud-init
elf: Make lookup_addr() respect symbol size
EC2 simulator: Add metadata API for Cassandra autoconfig
vfs: Fix lseek() boundary cases
tests: Test case for lseek()
Revert "tcp_output(): maximize LSO frames size"
Merge branch 'archlinux-build-fixes'
Revert "Adding libtools.so library"
scripts/test.py: Fix output alignment
Merge branch 'cloud-init-improvements'
Merge branch 'cli-port-commands'
Merge branch 'route-cache'
routecache: Remove debugging printout
mgmt: update to latest
Prasad Joshi (8):
fs: add openat systemcall
libc: add dirfd api
test: add openat testcase
libzfs: define unresolved devid_get function
libzfs: disable BSD specific geom_gettree call
zfs: skip character devices for ZFS pool lookup
zpool: fix pool import assert when name starts with number
zfs: ioctl(TRYIMPORT) must fail if config buf is insufficient
Raphael S. Carvalho (10):
x64: Introduce the instruction RDRAND
x64: random: provide RDRAND as a source of entropy for CSPRNG
random: Add debug messages for available hw sources
random: init and deinit live entropy sources
zfs: Fix spa import procedure for multi-vdev pools
mkfs: Automatic detection and insertion of additional vblk devices
random: fix random_harvest visibility for C++ source files
random: Print warning when no hw source of entropy is available
random: kill random_harvestq_cache
vfs: log file system unmounts
Renato Cunha (6):
sched: Add cpu_set_t as expected by linux programs
pthread: Add cpu_set_t support to pthread
pthread: Add pthread_attr_setaffinity_np and pthread_attr_getaffinity_np
pthread: Add stubbed versions of pthread_setaffinity_np and its get counterpart
tests: Add testing for pthread_attr_setaffinity_np and get counterpart
tests/tst-pthread-affinity: Remove bogus struct
Shlomi Livne (2):
Http Server: File System DF RESTful API
Test tst-poll.cc can not be runned twice
Takuya ASADA (13):
fs: add chown stub
math: Add __isinff
libc: Add __fdelt_chk
loader: add --ip on cmdline, static IP configuration
loader: add --defaultgw on cmdline, default gateway address
loader: add --nameserver on cmdline, nameserver address
isa-serial: Introduce early_init() to init driver ealier
fs: call abort on sys_panic, instead of kprintf
pci: Add VGA classcode
vga: use PCI classcode on probe function
pci: Add AHCI classcodes
ahci: use PCI classcode on probe function
ide: fix drive availability check
Tomasz Grabiec (71):
httpserver: compile using boost headers matching libraries we link with
httpserver: fix paths to misc.bin
httpserver: add missing stub for java_api::is_valid()
Extract httpserver into separate module
httpserver: add missing module.py
tests: fix pattern for os/version
httpserver: fix make rules for submodules
httpserver: do not override response code in request post-processing phase
httpserver: cleanup: remove unused parameter from reply400/reply500
httpserver: cleanup: make handle() return void
api: add RESTful API for controlling the sampling profiler
osvinit: fix compilation error
osvinit: express compilation-time dependency on httpserver with require()
httpserver: tests: the script is using the wrong command line
modules: locate httpserver in modules/
httpserver: fix module path used in module.py
httpserver: register /env API
httpserver: tests: do not ignore request failures
osvinit: update location of httpserver module
Bump up apps.git pointer
make: fix typo in .PHONY
make: clean all modules on 'make clean'
osvinit: fix linking of osvinit.so
net: respect MTU set via ioctl
random: fix includes
random: create random devices even when there's no virtio-rng
timer-set: make the assertion in expire() more verbose
httpserver: fix rule for swagger-ui
scripts: add EC2 metadata service simulator
bsd: wire get_cyclecount() to processor::ticks()
Bump up apps.git pointer
bsd: add missing include to random.h
ring_spsc: introduce emplace() method
core: introduce unordered_ring_mpsc
tests: add test for unordered_ring_mpsc
bsd: C++'ify random_harvestq.c
random: make random_harvest() non-blocking
random: collect entropy from interrupts
gdb: fix 'osv runqueue' on python3
bsd: make hash.h includeable from both C and C++ code
bsd: extract get_clockcycles() declaration to a header
bsd: reorder fields of struct random_adaptor to match initialization order
bsd: C++'ify source files realted to entropy harvesting
lockfree: align ring_spsc.emplace() to the STL standard
core: move unordered_ring_mpsc.hh form osv/ to lockfree/
Add utilities for working with exceptions
cloud-init: run before httpserver
cloud-init: client: make connection time out after 1 second
cloud-init: client: support for setting HTTP headers
cloud-init: client: introduce text() method for getting response contents
cloud-init: client: do not throw when response code is not 200
cloud-init: call load*() methods directly
cloud-init: turn 'once' flag to a standard behavior
cloud-init: add support for pluggable configuration modules
cloud-init: introduce meta-data service abstraction
cloud-init: load config from cloud's user data by default
cloud-init: link with boost_filesystem
cloud-init: add support for "files" syntax
cloud-init: clean up error handling
scripts: use EC2 tools from PATH
setup.py: append EC2_HOME/bin to PATH
setup.py: backup /etc/environment before override
setup.py: use standard EC2 packages on Ubuntu
dlfcn: fix corruption caused by dlclose()
epoll: make epoll_wait respect concurrent updates
tests: add test for epoll_ctl concurrent with epoll_wait
tests: make tst-epoll return non-zero exit code on error
tests: fix tst-sigwait.cc
modules/lua: fix the build
epoll: add support for EPOLLONESHOT
tests: epoll: add test for EPOLLONESHOT
Vlad Zolotarov (3):
percpu_xmit: Use a proper memory order in clear_pending()
tcp_output(): maximize LSO frames size
tcp_output(): maximize LSO frames size
Zifei Tong (13):
Add parentheses in print statement to support Py3K
Fix 'sign-compare' error in unordered_ring_mpsc.hh
build: change libgcc_s.a to libgcc.a
build: fix tst-static-thread-variable.so dependencies (again)
scripts: encode string before use it as buffer
httpserver: python3 support for json2code.py
scripts: add python3 support to test framework
scripts: add gcc 4.9 support to loader.py
build: use system gcc headers when build_env=host
mmu: show path and offset for file-mapped vma
elf: get rid of zero-length vmas
gdb: fix padding for short bits str
gdb: show offset/path for file mapped vma in 'osv mmap'
konya.vic@gmail.com (1):
pci: Fix wrong offsets of PCI configuration