Skip to content

Latest commit

 

History

History
258 lines (198 loc) · 9.65 KB

README.md

File metadata and controls

258 lines (198 loc) · 9.65 KB
Raw

TZF: a fast timezone finder for Go. Go Reference codecov

TZF is a fast timezone finder package designed for Go. It allows you to quickly find the timezone for a given latitude and longitude, making it ideal for geo queries and services such as weather forecast APIs. With optimized performance and two different data options, TZF is a powerful tool for any Go developer's toolkit.

Note

Here are some language or server which built with tzf or it's other language bindings:

Language or Sever Link Note
Go ringsaturn/tzf
Ruby HarlemSquirrel/tzf-rb build with tzf-rs
Rust ringsaturn/tzf-rs
Python ringsaturn/tzfpy build with tzf-rs
HTTP API ringsaturn/tzf-server build with tzf
HTTP API racemap/rust-tz-service build with tzf-rs
Redis Server ringsaturn/tzf-server build with tzf
Redis Server ringsaturn/redizone build with tzf-rs
JS via Wasm(browser only) ringsaturn/tzf-wasm build with tzf-rs
Online ringsaturn/tzf-web build with tzf-wasm

Quick Start

To start using TZF in your Go project, you first need to install the package:

go get github.com/ringsaturn/tzf

Then, you can use the following code to locate:

// Use about 150MB memory for init, and 60MB after GC.
package main

import (
	"fmt"

	"github.com/ringsaturn/tzf"
)

func main() {
	finder, err := tzf.NewDefaultFinder()
	if err != nil {
		panic(err)
	}
	fmt.Println(finder.GetTimezoneName(116.6386, 40.0786))
}

If you require a query result that is 100% accurate, use the following to locate:

// Use about 900MB memory for init, and 660MB after GC.
package main

import (
	"fmt"

	"github.com/ringsaturn/tzf"
	tzfrel "github.com/ringsaturn/tzf-rel"
	"github.com/ringsaturn/tzf/pb"
	"google.golang.org/protobuf/proto"
)

func main() {
	input := &pb.Timezones{}

	// Full data, about 83.5MB
	dataFile := tzfrel.FullData

	if err := proto.Unmarshal(dataFile, input); err != nil {
		panic(err)
	}
	finder, _ := tzf.NewFinderFromPB(input)
	fmt.Println(finder.GetTimezoneName(116.6386, 40.0786))
}

Best Practice

It's expensive to init tzf's Finder/FuzzyFinder/DefaultFinder, please consider reuse it or as a global var. Below is a global var example:

package main

import (
	"fmt"

	"github.com/ringsaturn/tzf"
)

var f tzf.F

func init() {
	var err error
	f, err = tzf.NewDefaultFinder()
	if err != nil {
		panic(err)
	}
}

func main() {
	fmt.Println(f.GetTimezoneName(116.3883, 39.9289))
	fmt.Println(f.GetTimezoneName(-73.935242, 40.730610))
}

CLI Tool

In addition to using TZF as a library in your Go projects, you can also use the tzf command-line interface (CLI) tool to quickly get the timezone name for a set of coordinates. To use the CLI tool, you first need to install it using the following command:

go install github.com/ringsaturn/tzf/cmd/tzf@latest

Once installed, you can use the tzf command followed by the latitude and longitude values to get the timezone name:

tzf -lng 116.3883 -lat 39.9289

Data

You can download the original data from https://github.com/evansiroky/timezone-boundary-builder.

The preprocessed protobuf data can be obtained from https://github.com/ringsaturn/tzf-rel, which has Go's embedded support. These files are Protocol Buffers messages for more efficient binary distribution, similar to Python wheels. You can view the pb/tzinfo.proto file or its HTML format documentation for information about the internal format.

The data pipeline for tzf can be illustrated as follows:

graph TD
    Raw[GeoJSON from evansiroky/timezone-boundary-builder]
    Full[Full: Probuf based data]
    Lite[Lite: smaller of Full data]
    Compressed[Compressed: Lite compressed via Polyline]
    Preindex[Tile based data]

    Finder[Finder: Polygon Based Finder]
    FuzzyFinder[FuzzyFinder: Tile based Finder]
    DefaultFinder[DefaultFinder: combine FuzzyFinder and Compressed Finder]

    Raw --> |cmd/geojson2tzpb|Full
    Full --> |cmd/reducetzpb|Lite
    Lite --> |cmd/compresstzpb|Compressed
    Lite --> |cmd/preindextzpb|Preindex

    Full --> |tzf.NewFinderFromPB|Finder
    Lite --> |tzf.NewFinderFromPB|Finder
    Compressed --> |tzf.NewFinderFromCompressed|Finder --> |tzf.NewDefaultFinder|DefaultFinder
    Preindex --> |tzf.NewFuzzyFinderFromPB|FuzzyFinder --> |tzf.NewDefaultFinder|DefaultFinder
Loading

The complete dataset (~80MB) can be used anywhere, but requires higher memory usage.

The lightweight dataset (~10MB) may not function optimally in some border areas.

You can observe points with different outcomes on this page.

If a slightly longer initialization time is tolerable, the compressed dataset (~5MB) derived from the lightweight dataset will be more suitable for binary distribution.

The pre-indexed dataset (~1.78MB) consists of multiple tiles. It is used within the DefaultFinder, which is built on FuzzyFinder, to reduce execution times of the raycasting algorithm.

I have written an article about the history of tzf, its Rust port, and its Rust port's Python binding; you can view it here.

Performance

The tzf package is intended for high-performance geospatial query services, such as weather forecasting APIs. Most queries can be returned within a very short time, averaging around 2000 nanoseconds.

Here is what has been done to improve performance:

  1. Using pre-indexing to handle most queries takes approximately 1000 nanoseconds.
  2. Using an RTree to filter candidate polygons, instead of iterating through all polygons, reduces the execution times of the Ray Casting algorithm.
  3. Using a finely-tuned Ray Casting algorithm package https://github.com/tidwall/geojson to verify whether a polygon contains a point.

That's all. There are no black magic tricks inside the tzf package.

The benchmark was conducted using version https://github.com/ringsaturn/tzf/releases/tag/v0.16.0

goos: darwin
goarch: arm64
pkg: github.com/ringsaturn/tzf
cpu: Apple M3 Max
BenchmarkDefaultFinder_GetTimezoneName_Random_WorldCities-16    	  823786	      1261 ns/op	      1000 ns/p50	      5000 ns/p90	      8000 ns/p99	       8 B/op	       0 allocs/op
BenchmarkFuzzyFinder_GetTimezoneName_Random_WorldCities-16      	 2239102	       572.1 ns/op	      1000 ns/p50	      1000 ns/p90	      1000 ns/p99	       8 B/op	       0 allocs/op
BenchmarkGetTimezoneName-16                                     	  423015	      2852 ns/op	      3000 ns/p50	      3000 ns/p90	      4000 ns/p99	       8 B/op	       0 allocs/op
BenchmarkGetTimezoneNameAtEdge-16                               	  399050	      3036 ns/op	      3000 ns/p50	      3000 ns/p90	      4000 ns/p99	       8 B/op	       0 allocs/op
BenchmarkGetTimezoneName_Random_WorldCities-16                  	  288864	      3867 ns/op	      4000 ns/p50	      6000 ns/p90	      8000 ns/p99	       8 B/op	       0 allocs/op
PASS
coverage: 65.5% of statements
ok  	github.com/ringsaturn/tzf	7.995s

Related Repos

Thanks