Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixes #2097 need a new way to load config file without decryption for… #2098

Merged
merged 4 commits into from
Feb 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,5 +10,5 @@ repos:
rev: 356f23a9acfcc8ac12697412e886483d2125cccd
hooks:
- id: keywordscan
args: ["--keywords=sunlife,cibc,eadp"]
args: ["--keywords=sunlife,cibc"]
types: ["text"]
1 change: 0 additions & 1 deletion NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -453,4 +453,3 @@ the Eclipse Foundation

under the terms of the GNU Lesser General Public License version 2.1
as published by the Free Software Foundation.

44 changes: 22 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,19 +16,19 @@ Light means lightweight, lightning fast and shedding light on how to program wit

## Why am I doing it

I had been working on the Java EE platforms since early 2000 and suffered the performance and productivity issues. In 2014, I realized that the IT industry was moving from Monolithic to Microservices and from on-premise data centers to the public clouds.
I had been working on the Java EE platforms since early 2000 and suffered the performance and productivity issues. In 2014, I realized that the IT industry was moving from Monolithic to Microservices and from on-premise data centers to the public clouds.

To reduce the production cost for my applications, I need to find a lightweight platform that has a small memory footprint, high throughput, and low latency in Java. Java EE and Spring/Spring Boot are too heavy to be considered. Other lightweight Java platforms all have different issues on the separation between the business logic and technical cross-cutting concerns. Moreover, they are not cloud-native or designed for container/cloud.
To reduce the production cost for my applications, I need to find a lightweight platform that has a small memory footprint, high throughput, and low latency in Java. Java EE and Spring/Spring Boot are too heavy to be considered. Other lightweight Java platforms all have different issues on the separation between the business logic and technical cross-cutting concerns. Moreover, they are not cloud-native or designed for container/cloud.

Without finding anything suitable, I started to build my open-source platform light-4j on top of the Undertow HTTP core by following the [principles](https://doc.networknt.com/about/principles/). Because my approach is very radical, and it is against products from big corporations, I have been attacked by some entities online and offline constantly. However, I am so convinced that my approach is the future direction and spend all my effort to continue the development. Today, a lot of companies are using light-4j on production with very positive feedback. It further encourages me to devote myself to the framework and to extend the platform to build an ecosystem.

For users who benefit from the platform, you can [become a sponsor](https://github.com/sponsors/stevehu) for me so that I can put all my effort into the open-source projects.
For users who benefit from the platform, you can [become a sponsor](https://github.com/sponsors/stevehu) for me so that I can put all my effort into the open-source projects.

## Why use this framework

### Fast and small memory footprint to lower production cost.

It is 44 times faster than the most popular microservices platform Spring Boot embedded Tomcat and use only 1/5 of memory. Here are the [benchmark](https://github.com/networknt/microservices-framework-benchmark) results compare with Spring Boot and other microservices frameworks. Here is the third party [comparison](https://www.techempower.com/benchmarks/) with other Web frameworks.
It is 44 times faster than the most popular microservices platform Spring Boot embedded Tomcat and use only 1/5 of memory. Here are the [benchmark](https://github.com/networknt/microservices-framework-benchmark) results compare with Spring Boot and other microservices frameworks. Here is the third party [comparison](https://www.techempower.com/benchmarks/) with other Web frameworks.

### Provide an embedded gateway to address cross-cutting concerns.
* Plugin architecture for startup/shutdown hooks and middleware components
Expand All @@ -41,15 +41,15 @@ It is 44 times faster than the most popular microservices platform Spring Boot e
* Audit to dump important info or entire request and response.
* Body parser to support different content types
* Standardized response code and messages from the configuration file
* Externalized configuration for all modules for the dockerized environment
* Externalized configuration for all modules for the dockerized environment
* CORS pre-flight handler for SPA (Angular or React) from another domain
* Rate limiting for services that exposed outside to the Internet
* Service registry and discovery support direct, Consul and Zookeeper
* Client-side discovery and load balance to eliminate proxies
* A client module that is tightly integrated with Light-OAuth2 and supports traceability

### Design and Test driven development to increase productivity
Design OpenAPI specification and generate the service from it. The specification is also
Design OpenAPI specification and generate the service from it. The specification is also
part of the framework to drive security verification and request validation at runtime.

Unit/End-to-End test stubs are generated to enable test driven approach for quality product.
Expand All @@ -72,14 +72,14 @@ integration to production.

All the open sourced frameworks are built in Java and we are working on Nodejs framework internally.
In the future, we might provide Golang framework as well and all them are sharing the same eco-system
and market place.
and market place.


### OAuth2 server, portal and services to form ecosystem

[OAuth2 Server](https://github.com/networknt/light-oauth2) for security and [Portal](https://github.com/networknt/light-portal)
for production monitor and management. The portal is also a marketplace to link clients and services
together.
for production monitor and management. The portal is also a marketplace to link clients and services
together.


## Getting Started
Expand All @@ -89,30 +89,30 @@ There are two ways to start your project:
### Light-codegen generator

You can use [light-codegen](https://github.com/networknt/light-codegen) to generate a working project.
Currently, it supports light-rest-4j, light-graphql-4j, light-hybrid-server-4j and light-hybrid-service-4j.
light-eventuate code generator is coming.
Currently, it supports light-rest-4j, light-graphql-4j, light-hybrid-server-4j and light-hybrid-service-4j.
light-eventuate code generator is coming.

The light-codegen project README.md describes four ways to use the generator with examples.

* Clone and build the light-codgen and use the codegen-cli command line utility
* Use docker image networknt/light-codegen to run the codegen-cli command line utility
* Use generate.sh from [model-config](https://github.com/networknt/model-config) repo to generate projects based on conventions.
* Generate code from web site with codegen-web API. (API is ready but UI needs to be built)
* Use generate.sh from [model-config](https://github.com/networknt/model-config) repo to generate projects based on conventions.
* Generate code from web site with codegen-web API. (API is ready but UI needs to be built)


### Starting from an example project

The other way to start your project is to copy from [light-example-4j](https://github.com/networknt/light-example-4j).

You can find the description of these [examples](https://www.networknt.com/example/)
You can find the description of these [examples](https://www.networknt.com/example/)

Also, there are some [tutorials](https://www.networknt.com/tutorial/)
Also, there are some [tutorials](https://www.networknt.com/tutorial/)


## Debugging

[To run/debug from IDE](https://www.networknt.com/tutorial/common/debug/), you need to
configure a Java application with main class "com.networknt.server.Server" and working
[To run/debug from IDE](https://www.networknt.com/tutorial/common/debug/), you need to
configure a Java application with main class "com.networknt.server.Server" and working
directory is your project folder. There is no container and you are working on just a standalone
Java application.

Expand Down Expand Up @@ -144,15 +144,15 @@ kill -s TERM <pid>
The server has a shutdown hook and the above command allow it to clean up. For example,
complete in-flight requests and close the database connections etc. If service registry
and discovery is used, then the server will send shutdown event to service registry and
keep processing requests for 30 seconds until all clients refreshes their local cache
before shutting down.
keep processing requests for 30 seconds until all clients refreshes their local cache
before shutting down.

## Appreciation

- Light-4j has been optimized by using open source license of [JProfiler](http://www.ej-technologies.com/products/jprofiler/overview.html)
from [ej-technologies](http://www.ej-technologies.com/).
- Light-4j has been optimized by using open source license of [JProfiler](http://www.ej-technologies.com/products/jprofiler/overview.html)
from [ej-technologies](http://www.ej-technologies.com/).

## License

Light-4j and all light-*-4j frameworks are available under the Apache 2.0 license. See the [LICENSE](https://www.apache.org/licenses/LICENSE-2.0)
Light-4j and all light-*-4j frameworks are available under the Apache 2.0 license. See the [LICENSE](https://www.apache.org/licenses/LICENSE-2.0)
file for more info.
2 changes: 1 addition & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@

## Reporting a Vulnerability

To report a security issue, please email stevehu@gmail.com with a description of the issue, the steps you took to create the issue, affected versions, and, if known, mitigations for the issue. Our vulnerability management team will acknowledge receiving your email within three working days. This project follows a 90-day disclosure timeline.
To report a security issue, please email stevehu@gmail.com with a description of the issue, the steps you took to create the issue, affected versions, and, if known, mitigations for the issue. Our vulnerability management team will acknowledge receiving your email within three working days. This project follows a 90-day disclosure timeline.
4 changes: 2 additions & 2 deletions api-key/src/main/java/com/networknt/apikey/ApiKeyHandler.java
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ public void register() {
if(!config.hashEnabled) {
masks.add("apiKey");
}
ModuleRegistry.registerModule(ApiKeyConfig.CONFIG_NAME, ApiKeyHandler.class.getName(), Config.getInstance().getJsonMapConfigNoCache(ApiKeyConfig.CONFIG_NAME), masks);
ModuleRegistry.registerModule(ApiKeyConfig.CONFIG_NAME, ApiKeyHandler.class.getName(), Config.getNoneDecryptedInstance().getJsonMapConfigNoCache(ApiKeyConfig.CONFIG_NAME), masks);
}

@Override
Expand All @@ -86,7 +86,7 @@ public void reload() {
if(!config.hashEnabled) {
masks.add("apiKey");
}
ModuleRegistry.registerModule(ApiKeyConfig.CONFIG_NAME, ApiKeyHandler.class.getName(), Config.getInstance().getJsonMapConfigNoCache(ApiKeyConfig.CONFIG_NAME), masks);
ModuleRegistry.registerModule(ApiKeyConfig.CONFIG_NAME, ApiKeyHandler.class.getName(), Config.getNoneDecryptedInstance().getJsonMapConfigNoCache(ApiKeyConfig.CONFIG_NAME), masks);
if(logger.isInfoEnabled()) logger.info("ApiKeyHandler is reloaded.");
}

Expand Down
2 changes: 1 addition & 1 deletion api-key/src/test/resources/config/apikey.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,4 @@ pathPrefixAuths:
apiKey: xyz
- pathPrefix: /test2
headerName: x-apikey
apiKey: CRYPT:3ddd6c8b9bf2afc24d1c94af1dffd518:1bf0cafb19c53e61ddeae626f8906d43
apiKey: CRYPT:3ddd6c8b9bf2afc24d1c94af1dffd518:1bf0cafb19c53e61ddeae626f8906d43
2 changes: 1 addition & 1 deletion audit-config/src/main/java/module-info.j
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ module com.networknt.audit {
requires com.fasterxml.jackson.core;
requires org.slf4j;
requires java.logging;
}
}
2 changes: 1 addition & 1 deletion audit-config/src/test/resources/logback-test.xml
Original file line number Diff line number Diff line change
Expand Up @@ -71,4 +71,4 @@
<appender-ref ref="audit"/>
</logger>

</configuration>
</configuration>
Original file line number Diff line number Diff line change
@@ -1 +1 @@
mock-maker-inline
mock-maker-inline
2 changes: 1 addition & 1 deletion audit-config/src/test/resources/values.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ audit.headers:
# This format is more friendly with the config server.
audit.audit: client_id, user_id, scope_client_id, endpoint, serviceId, requestBody, responseBody, queryParameters, pathParameters, requestCookies, status
# This is another format that is friendly with the config server
# audit.audit: ["client_id","user_id","scope_client_id","endpoint","serviceId","responseBody","queryParameters","pathParameters","requestCookies","status"]
# audit.audit: ["client_id","user_id","scope_client_id","endpoint","serviceId","responseBody","queryParameters","pathParameters","requestCookies","status"]
4 changes: 2 additions & 2 deletions audit/src/main/java/com/networknt/audit/AuditHandler.java
Original file line number Diff line number Diff line change
Expand Up @@ -362,13 +362,13 @@ public boolean isEnabled() {

@Override
public void register() {
ModuleRegistry.registerModule(AuditConfig.CONFIG_NAME, AuditHandler.class.getName(), config.getMappedConfig(), null);
ModuleRegistry.registerModule(AuditConfig.CONFIG_NAME, AuditHandler.class.getName(), Config.getNoneDecryptedInstance().getJsonMapConfigNoCache(AuditConfig.CONFIG_NAME), null);
}

@Override
public void reload() {
config.reload();
ModuleRegistry.registerModule(AuditConfig.CONFIG_NAME, AuditHandler.class.getName(), config.getMappedConfig(), null);
ModuleRegistry.registerModule(AuditConfig.CONFIG_NAME, AuditHandler.class.getName(), Config.getNoneDecryptedInstance().getJsonMapConfigNoCache(AuditConfig.CONFIG_NAME), null);
if(logger.isInfoEnabled()) logger.info("AuditHandler is reloaded.");
}
}
2 changes: 1 addition & 1 deletion audit/src/main/java/module-info.j
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ module com.networknt.audit {
requires com.fasterxml.jackson.core;
requires org.slf4j;
requires java.logging;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -78,4 +78,4 @@ public boolean isEnabled() {
public void register() {
}

}
}
2 changes: 1 addition & 1 deletion audit/src/test/resources/logback-test.xml
Original file line number Diff line number Diff line change
Expand Up @@ -71,4 +71,4 @@
<appender-ref ref="audit"/>
</logger>

</configuration>
</configuration>
Original file line number Diff line number Diff line change
@@ -1 +1 @@
mock-maker-inline
mock-maker-inline
2 changes: 1 addition & 1 deletion audit/src/test/resources/server.yml
Original file line number Diff line number Diff line change
@@ -1 +1 @@
serviceId: com.networknt.petstore-1.0.0
serviceId: com.networknt.petstore-1.0.0
2 changes: 1 addition & 1 deletion audit/src/test/resources/values.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ audit.headers:
# This format is more friendly with the config server.
audit.audit: client_id, user_id, scope_client_id, endpoint, serviceId, requestBody, responseBody, queryParameters, pathParameters, requestCookies, status
# This is another format that is friendly with the config server
# audit.audit: ["client_id","user_id","scope_client_id","endpoint","serviceId","responseBody","queryParameters","pathParameters","requestCookies","status"]
# audit.audit: ["client_id","user_id","scope_client_id","endpoint","serviceId","responseBody","queryParameters","pathParameters","requestCookies","status"]
2 changes: 1 addition & 1 deletion balance/src/main/java/module-info.j
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@ module com.networknt.balance {
requires com.networknt.utility;

requires org.slf4j;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,8 @@ public void testSelect() throws Exception {
URL url = loadBalance.select(urls, "serviceId", "tag", null);
Assert.assertEquals(url, URLImpl.valueOf("http://127.0.0.1:8081/v1"));
}
@Test

@Test
public void testSelectFirstThenRoundRobin() throws Exception{
List<URL> urls = new ArrayList<>();
urls.add(new URLImpl("http", "127.0.0.10", 8081, "v1", new HashMap<String, String>()));
Expand All @@ -62,7 +62,7 @@ public void testSelectFirstThenRoundRobin() throws Exception{
URL url = loadBalance.select(urls, "serviceId", "tag", null);
Assert.assertTrue(urls.contains(url));
}

@Test
public void testSelectWithEmptyList() throws Exception {
List<URL> urls = new ArrayList<>();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ public void testSelectWithEmptyList() throws Exception {

/**
* There is a chance that both values are the same but it should be very rare. Given this possibility,
* ignore this test unless we need it.
* ignore this test unless we need it.
*/
@Ignore
@Test
Expand Down
2 changes: 1 addition & 1 deletion balance/src/test/resources/logback-test.xml
Original file line number Diff line number Diff line change
Expand Up @@ -46,4 +46,4 @@
<appender-ref ref="log"/>
</logger>

</configuration>
</configuration>
Original file line number Diff line number Diff line change
Expand Up @@ -332,15 +332,15 @@ public void register() {
// As passwords are in the config file, we need to mask them.
List<String> masks = new ArrayList<>();
masks.add("password");
ModuleRegistry.registerModule(BasicAuthConfig.CONFIG_NAME, BasicAuthHandler.class.getName(), Config.getInstance().getJsonMapConfigNoCache(BasicAuthConfig.CONFIG_NAME), masks);
ModuleRegistry.registerModule(BasicAuthConfig.CONFIG_NAME, BasicAuthHandler.class.getName(), Config.getNoneDecryptedInstance().getJsonMapConfigNoCache(BasicAuthConfig.CONFIG_NAME), masks);
}

@Override
public void reload() {
config.reload();
List<String> masks = new ArrayList<>();
masks.add("password");
ModuleRegistry.registerModule(BasicAuthConfig.CONFIG_NAME, BasicAuthHandler.class.getName(), Config.getInstance().getJsonMapConfigNoCache(BasicAuthConfig.CONFIG_NAME), masks);
ModuleRegistry.registerModule(BasicAuthConfig.CONFIG_NAME, BasicAuthHandler.class.getName(), Config.getNoneDecryptedInstance().getJsonMapConfigNoCache(BasicAuthConfig.CONFIG_NAME), masks);
if(logger.isInfoEnabled()) logger.info("BasicAuthHandler is reloaded.");
}
}
2 changes: 1 addition & 1 deletion basic-auth/src/main/java/module-info.j
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ module com.networknt.basic.auth {
requires undertow.core;
requires org.slf4j;
requires java.logging;
}
}
2 changes: 1 addition & 1 deletion basic-auth/src/test/resources/config/values.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ ldap.credential: CRYPT:0754fbc37347c136be7725cbf62b6942:71756e13c2400985d0402ed6
ldap.searchFilter: (&(samaccountname=%s)(objectClass=person))
ldap.searchBase: ''

client.verifyHostname: false
client.verifyHostname: false
2 changes: 1 addition & 1 deletion body-config/src/main/java/module-info.j
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ module com.networknt.body {
requires org.slf4j;
requires java.logging;
requires com.fasterxml.jackson.core;
}
}
4 changes: 2 additions & 2 deletions body/src/main/java/com/networknt/body/BodyHandler.java
Original file line number Diff line number Diff line change
Expand Up @@ -208,13 +208,13 @@ public boolean isEnabled() {

@Override
public void register() {
ModuleRegistry.registerModule(BodyConfig.CONFIG_NAME, BodyHandler.class.getName(), Config.getInstance().getJsonMapConfigNoCache(BodyConfig.CONFIG_NAME), null);
ModuleRegistry.registerModule(BodyConfig.CONFIG_NAME, BodyHandler.class.getName(), Config.getNoneDecryptedInstance().getJsonMapConfigNoCache(BodyConfig.CONFIG_NAME), null);
}

@Override
public void reload() {
config.reload();
ModuleRegistry.registerModule(BodyConfig.CONFIG_NAME, BodyHandler.class.getName(), config.getMappedConfig(), null);
ModuleRegistry.registerModule(BodyConfig.CONFIG_NAME, BodyHandler.class.getName(), Config.getNoneDecryptedInstance().getJsonMapConfigNoCache(BodyConfig.CONFIG_NAME), null);
if(logger.isInfoEnabled()) logger.info("BodyHandler is reloaded.");
}
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package com.networknt.body;

import com.networknt.config.Config;
import com.networknt.handler.BuffersUtils;
import com.networknt.handler.MiddlewareHandler;
import com.networknt.handler.RequestInterceptor;
Expand Down Expand Up @@ -188,13 +189,13 @@ public boolean isEnabled() {

@Override
public void register() {
ModuleRegistry.registerModule(BodyConfig.CONFIG_NAME, RequestBodyInterceptor.class.getName(), config.getMappedConfig(), null);
ModuleRegistry.registerModule(BodyConfig.CONFIG_NAME, RequestBodyInterceptor.class.getName(), Config.getNoneDecryptedInstance().getJsonMapConfigNoCache(BodyConfig.CONFIG_NAME), null);
}

@Override
public void reload() {
config.reload();
ModuleRegistry.registerModule(BodyConfig.CONFIG_NAME, RequestBodyInterceptor.class.getName(), config.getMappedConfig(), null);
ModuleRegistry.registerModule(BodyConfig.CONFIG_NAME, RequestBodyInterceptor.class.getName(), Config.getNoneDecryptedInstance().getJsonMapConfigNoCache(BodyConfig.CONFIG_NAME), null);
if (LOG.isInfoEnabled())
LOG.info("RequestBodyInterceptor is reloaded.");
}
Expand Down
Loading