Skip to content

Commit

Permalink
Performance improvement on LLVM build time (#20)
Browse files Browse the repository at this point in the history
Summary:
Do a shallow clone using `--shallow-since` flag
in order to spend less time cloning the LLVM repo.
The LLVM repo is huge, which leads to a 400+ MB
directory, and it takes a reasonable time to clone
the repository.

Adding a `--shallow-since` flag using the date
before the desired revision led to a performance
improvement of 4x cloning the repo.

The 8 Oct 2018 date is the day before the desired
revision specified in the code.

## My local tests:

**Without shallow clone:**

```
renato@renato-data-tmp:/tmp$ time git clone https://github.com/llvm-mirror/llvm.git
Cloning into 'llvm'...
remote: Enumerating objects: 166, done.
remote: Counting objects: 100% (166/166), done.
remote: Compressing objects: 100% (134/134), done.
remote: Total 1726648 (delta 52), reused 65 (delta 32), pack-reused 1726482
Receiving objects: 100% (1726648/1726648), 425.35 MiB | 9.27 MiB/s, done.
Resolving deltas: 100% (1418304/1418304), done.
Checking out files: 100% (38646/38646), done.

real	3m40.953s
user	4m38.279s
sys	0m10.589s
```

**With shallow clone:**

```
renato@renato-data-tmp:/tmp$ time git clone --shallow-since 2018-10-08 https://github.com/llvm-mirror/llvm.git llvm-2
Cloning into 'llvm-2'...
remote: Enumerating objects: 168108, done.
remote: Counting objects: 100% (168108/168108), done.
remote: Compressing objects: 100% (61767/61767), done.
Receiving objects: 100% (168108/168108), 91.62 MiB | 9.57 MiB/s, done.
remote: Total 168108 (delta 117253), reused 137482 (delta 104959), pack-reused 0
Resolving deltas: 100% (117253/117253), done.
Checking connectivity: 168108, done.
Checking out files: 100% (38646/38646), done.

real	0m52.412s
user	0m39.649s
sys	0m3.658s
```
Pull Request resolved: #20

Differential Revision: D16242893

Pulled By: dulinriley

fbshipit-source-id: 89c155bea55658c0faab3c5c9886254e2660698f
  • Loading branch information
renatoalencar authored and facebook-github-bot committed Jul 13, 2019
1 parent 5ff757c commit 4233d4f
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion utils/build_llvm.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,12 @@
import sys


# It references the commit day so we can shallow clone
# and still manage to checkout this specific revision.
# NOTE: The revision date must be the day before the
# actual commit date.
_LLVM_REV = "c179d7b006348005d2da228aed4c3c251590baa3"
_LLVM_REV_DATE = "2018-10-08"


def parse_args():
Expand Down Expand Up @@ -166,7 +171,7 @@ def main():
print("Cloning LLVM into {}".format(args.llvm_src_dir))
run_command(
git
+ ["clone", "https://github.com/llvm-mirror/llvm.git", args.llvm_src_dir]
+ ["clone", "--shallow-since", _LLVM_REV_DATE, "https://github.com/llvm-mirror/llvm.git", args.llvm_src_dir]
)

# Checkout a specific revision in LLVM.
Expand Down

0 comments on commit 4233d4f

Please sign in to comment.