Reproducible build: introduce a tool to help reproducible build #268

longlongyang · 2022-05-15T14:42:41Z

Introduce a reproducible tool, td-reproducible-tool, to help reproducible
Add strip = "symbols" to cargo.toml in root workspace to remove symbols

Signed-off-by: Longlong Yang longlong.yang@intel.com

gaojiaqi7 · 2022-05-16T16:21:52Z

The CI failure caused by secure boot verification is due to strip = "symbols", there may be some hidden optimizations which are not compatible with the new setting

jiangliu · 2022-05-19T04:19:01Z

td-reproducible-tool/readme.md

+```
+
+Solution: <br>
+


Could 2, 3, 4 also be solved by using build container to normalize the build env?

It is based upon the definition of "reproducible build".

In our definition, "reproducible build" means
A) Release build
B) Build with same code and version
C) Build with same compiler and version
D) Build at different time.
E) Build at different directory/path <==

If we normalize the build, then E) is not requirement, then we don't need 2/3/4.

The final design:
1 and 2 are handled by build option.
3 and 4 are handled by post-build tool as an option.
5. Timestamp is always cleared.

td-reproducible-tool/readme.md

jyao1 · 2022-05-27T01:45:55Z

td-reproducible-tool/src/main.rs

+    // check out CARGO_HOME and RUSTUP_HOME
+    cargo_home = if cargo_home == "" {
+        if env::var("CARGO_HOME").is_err() {
+            panic!("Neither --cargo_home nor system environment for \"CARGO_HOME\" is found! ")


Can we make it optional?

Can we make it optional?

It is optional now, please check the latest code change.

jyao1 · 2022-05-27T01:45:55Z

td-reproducible-tool/src/main.rs

+    // check out CARGO_HOME and RUSTUP_HOME
+    cargo_home = if cargo_home == "" {
+        if env::var("CARGO_HOME").is_err() {
+            panic!("Neither --cargo_home nor system environment for \"CARGO_HOME\" is found! ")


Can we make it optional?

Can we make it optional?

It is optional now, please check the latest code change.

jyao1 · 2022-05-27T01:46:09Z

td-reproducible-tool/src/main.rs

+
+    rustup_home = if rustup_home == "" {
+        if env::var("RUSTUP_HOME").is_err() {
+            panic!("Neither --rustup_home nor system environment for \"RUSTUP_HOME\" is found! ")


Can we make it optional?

Can we make it optional?

It is optional now, please check the latest code change.

jyao1 · 2022-05-27T01:46:09Z

td-reproducible-tool/src/main.rs

+
+    rustup_home = if rustup_home == "" {
+        if env::var("RUSTUP_HOME").is_err() {
+            panic!("Neither --rustup_home nor system environment for \"RUSTUP_HOME\" is found! ")


Can we make it optional?

It is optional now, please check the latest code change.

jyao1 · 2022-05-27T07:58:28Z

td-reproducible-tool/src/main.rs

+
+    rustup_home = if rustup_home == "" {
+        if env::var("RUSTUP_HOME").is_err() {
+            panic!("Neither --rustup_home nor system environment for \"RUSTUP_HOME\" is found!\n")


still panic?

still panic?

td-shim/td-reproducible-tool/src/main.rs

Lines 138 to 151 in c3a977b

// No more action is needed if strip_path is not specified.

if !strip_path {

println!(

"INFO: -s or --strip_path is not specified, Skipping strip related rust file path."

);

return Ok(());

}

// Check out CARGO_HOME and RUSTUP_HOME, proceed to strip rust file path.

cargo_home = if cargo_home == "" {

if env::var("CARGO_HOME").is_err() {

panic!("Neither --cargo_home nor system environment for \"CARGO_HOME\" is found!\n")

} else {

env::var("CARGO_HOME").unwrap()

If need to strip file path(-s or --strip_path is pecified), then this two is required to set. We can't strip file path without this file path prefix known to us. So I just panic here if not found the path prefix.

OK. I did not found -s or --strip_path in readme.

Please add it.

OK. I did not found -s or --strip_path in readme.

Please add it.

Sorry for forgot to update readme. Now added, please check the latest readme.md

jyao1 · 2022-05-27T07:58:49Z

td-reproducible-tool/src/main.rs

+    // Check out CARGO_HOME and RUSTUP_HOME, proceed to strip rust file path.
+    cargo_home = if cargo_home == "" {
+        if env::var("CARGO_HOME").is_err() {
+            panic!("Neither --cargo_home nor system environment for \"CARGO_HOME\" is found!\n")


still panic?

1. Introduce a reproducible tool, td-reproducible-tool, to help reproducible 2. Add strip = "symbols" to cargo.toml in root workspace to remove symbols Signed-off-by: Longlong Yang <longlong.yang@intel.com>

liuw1 · 2022-05-28T13:18:42Z

I validated reproducible build with the latest code, there are still some differences. And the secure boot issue was found in the latest CI test.

jyao1 · 2022-05-28T13:21:04Z

@liuw1 , please file a new issue, or reopen the old issue.

longlongyang · 2022-05-29T13:10:50Z

I validated reproducible build with the latest code, there are still some differences. And the secure boot issue was found in the latest CI test.

This position is the where TimeDateStamp sit at, Did you run reproducible tool after every binary built?
And are you sure you observed the secure boot issue using the commit newer or equal to commit#fc436aabae?

jyao1 · 2022-05-29T13:51:16Z

@longlongyang , I think we need update readme.md to add step for reproducible tool.
Otherwise, people may forget that step.

longlongyang · 2022-05-29T13:52:43Z

@longlongyang , I think we need update readme.md to add step for reproducible tool. Otherwise, people may forget that step.

Okay, will update.

jyao1 mentioned this pull request May 16, 2022

Need to upgrade the toolchain to 1.59 release to accept "strip" option in release profile #269

Closed

jyao1 closed this May 16, 2022

jyao1 reopened this May 16, 2022

jyao1 approved these changes May 16, 2022

View reviewed changes

gaojiaqi7 closed this May 17, 2022

gaojiaqi7 reopened this May 17, 2022

gaojiaqi7 closed this May 17, 2022

gaojiaqi7 reopened this May 17, 2022

longlongyang mentioned this pull request May 18, 2022

[Bug] secure_boot_verify_payload fails with InvalidSignature when using ECDSA_NIST_P384_SHA384 crypto suite #272

Closed

longlongyang force-pushed the main branch from d88ad7c to d9c53cb Compare May 19, 2022 02:28

jiangliu reviewed May 19, 2022

View reviewed changes

longlongyang force-pushed the main branch from d9c53cb to ba6d868 Compare May 23, 2022 07:36

jyao1 reviewed May 23, 2022

View reviewed changes

td-reproducible-tool/readme.md Show resolved Hide resolved

longlongyang force-pushed the main branch from ba6d868 to 8c2b0ab Compare May 23, 2022 15:44

jyao1 reviewed May 27, 2022

View reviewed changes

longlongyang force-pushed the main branch 2 times, most recently from 47de45f to c3a977b Compare May 27, 2022 02:29

jyao1 reviewed May 27, 2022

View reviewed changes

longlongyang force-pushed the main branch from c3a977b to b0c3fde Compare May 27, 2022 09:42

Reproducible build: introduce a tool to help reproducible build

4c758bb

1. Introduce a reproducible tool, td-reproducible-tool, to help reproducible 2. Add strip = "symbols" to cargo.toml in root workspace to remove symbols Signed-off-by: Longlong Yang <longlong.yang@intel.com>

longlongyang force-pushed the main branch from b0c3fde to 4c758bb Compare May 27, 2022 09:45

jyao1 merged commit fc436aa into confidential-containers:main May 27, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reproducible build: introduce a tool to help reproducible build #268

Reproducible build: introduce a tool to help reproducible build #268

longlongyang commented May 15, 2022

gaojiaqi7 commented May 16, 2022

jiangliu May 19, 2022

jyao1 May 19, 2022 •

edited

Loading

jyao1 May 27, 2022

jyao1 May 27, 2022

longlongyang May 27, 2022

jyao1 May 27, 2022

longlongyang May 27, 2022

jyao1 May 27, 2022

longlongyang May 27, 2022

jyao1 May 27, 2022

longlongyang May 27, 2022

jyao1 May 27, 2022

longlongyang May 27, 2022 •

edited

Loading

jyao1 May 27, 2022

longlongyang May 27, 2022 •

edited

Loading

jyao1 May 27, 2022

liuw1 commented May 28, 2022

jyao1 commented May 28, 2022

longlongyang commented May 29, 2022

jyao1 commented May 29, 2022

longlongyang commented May 29, 2022

	// No more action is needed if strip_path is not specified.
	if !strip_path {
	println!(
	"INFO: -s or --strip_path is not specified, Skipping strip related rust file path."
	);
	return Ok(());
	}

	// Check out CARGO_HOME and RUSTUP_HOME, proceed to strip rust file path.
	cargo_home = if cargo_home == "" {
	if env::var("CARGO_HOME").is_err() {
	panic!("Neither --cargo_home nor system environment for \"CARGO_HOME\" is found!\n")
	} else {
	env::var("CARGO_HOME").unwrap()

		```

		Solution: <br>

Reproducible build: introduce a tool to help reproducible build #268

Reproducible build: introduce a tool to help reproducible build #268

Conversation

longlongyang commented May 15, 2022

gaojiaqi7 commented May 16, 2022

Choose a reason for hiding this comment

jyao1 May 19, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

longlongyang May 27, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

longlongyang May 27, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

liuw1 commented May 28, 2022

jyao1 commented May 28, 2022

longlongyang commented May 29, 2022

jyao1 commented May 29, 2022

longlongyang commented May 29, 2022

jyao1 May 19, 2022 •

edited

Loading

longlongyang May 27, 2022 •

edited

Loading

longlongyang May 27, 2022 •

edited

Loading