Demo 99% Restored

New RPC Structure
Reduced packet processing
Reduced packet size
Whois Certificates Started
Reduced Domain Certificates
Frames now used in packet messages
conversion to micro packet system in progress

README.md

@@ -1,13 +1,13 @@
-<h2 align="center">
+<h1 align="center">
     <a href="uw://universalweb.io">UNIVERSAL WEB</a>
-</h2>
+</h1>
 <p align="center">
 | <a href="https://github.com/universalweb/Whitepaper">WHITEPAPER (OUTDATED)</a> |
 </p>
 <hr />
 <h1 align="center">UW://</h1>
 <p align="center">
-    REIMAGINE THE WEB
+    REIMAGINE|REMAKE|REBUILD - THE WEB
 </p>
 
 <h5 align="center">| <a href="https://twitter.com/tommarchi">LEAD DEV TWITTER</a> |</h5>
@@ -21,9 +21,7 @@
 </p>
 
 <h3>Perspective</h3>
-<p>The World Wide Web has evolved for decades but its general design, limitations, & components have largely remained. Our needs & desires have rapidly outgrown the initial design & purpose of the Web. What if the Web was being reimagined & built today? What would be different? Would components change, be removed, or turn into something new? Take the decades of lessons learned combine it with modern technology and create a Web designed for the future.</p>
-
-<p>The current definition for Web3 isn't a solution it doesn't solve root issues. Therefore we don't consider the UW a Web3 network by current definitions and comparisons.</p>
+<p>The World Wide Web has evolved for decades but its general design, limitations, & components have largely remained. Our needs & desires have rapidly outgrown the initial design & purpose of the Web. What if the Web was reimagined today?</p>
 
 <h4>Goals</h4>
 <p>The objective of the Universal Web is to create a viable modern replacement for the Web today.</p>
@@ -32,7 +30,7 @@
 <p>Not exactly. If the definition of Web3 is a fully decentralized Web then The Universal Web is not Web3. If Web3 is the World Wide Web plus some decentralized apps then the UW is not Web3. If Web3 is just the general idea of mixing something like the Web with optional and or native integration of decentralized parts then it’s much closer to what the UW is.</p>
 
 <h4>Where's the name from?</h4>
-<p>The Universal Web name was chosen because of its natural evolution of the name WE USE FOR THE web today being The World Wide Web. The idea being let’s build a Web that we could see being used on Mars or in space.</p>
+<p>The Universal Web's name was chosen because of its natural evolution of the name The World Wide Web.</p>
 
 <h4>Internet vs Web</h4>
 <p>The Web is all the software bits that sits on-top of the Internet. The Universal Web at least for now is only concerned with software no special hardware is required. Users just need software to access the Universal Web and Servers just need software to setup a Universal Web server.</p>

howAreURLParametersHandled.md

@@ -1,4 +1,4 @@
-# How are URL parameters handled compared to HTTP?
+# How are URL parameters handled compared to HTTP? EXPERIMENTAL NOT FINAL
 
 Instead of URL paramaters being part of the Request-line they are turned into an structured object with binary support.
 This enables: the ability for complex URL parameter requests, request are safer, no need to sanative, doesn't require URL encoding went sent, doesn't require URL decoding, less client side code, less server side code, less code complexity, easier to type, can support binary parameters, and are easier to understand.

rpcCodes.md

@@ -23,23 +23,21 @@ RPC Codes are numbers that correspond to methods, functions, and or procedures.
 
 ### Message RPC Codes
 
-- INTRO [0, false]
-- 1
-- 2
-- 3
-- 4
-- 5
-- 6
-- 7
+- 0 INTRO [false, 0]
+- 1 SETUP [1]
+- 2 PATH READY [2]
+- 3 PARAMETERS READY [3]
+- 4 HEAD READY [4]
+- 5 DATA READY [5]
+- 6 PATH
+- 7 PARAMETERS
+- 8 HEAD
+- 9 DATA
+- 10 Close - END
+- 11 Error
 
 ### Message & Frame Format
 
 Messages hold a frame or frames. A packet can only hold one message but it can include multiple frames.
 
 [stream id, rpc, ...values]
-
-Same stream ID with multiple frame types
-[stream id, [rpc, ...values]]
-
-Example of multiple frames with different stream ids in a single message/packet.
-[[stream id, rpc, ...values], [stream id, rpc, ...values]]

scripts/certificates.js

@@ -5,11 +5,11 @@ const dirname = currentPath(import.meta);
 const domainProfile = await createProfile({
 	template: {
 		ephemeral: {
-			nft: {
-				network: 'eth',
-				contract: '0x0000000',
-				id: '0x0000000',
-			},
+			// nft: {
+			// 	network: 'eth',
+			// 	contract: '0x0000000',
+			// 	id: '0x0000000',
+			// },
 			// version is the version number of uw://
 			// certificate version
 			version: 1,
@@ -50,12 +50,13 @@ const domainProfile = await createProfile({
 			// maxPacketPathSize: 1200,
 			// maxPacketParametersSize: 1200,
 			url: 'universal.web',
-			domain: 'universal',
-			extension: 'web',
+			// domain: 'universal',
+			// extension: 'web',
 			// The ip, ipv4, & ipv6 properties are used to determine the IP address of the server.
 			// The ip property can be either ipv4 or ipv6.
 			// ipv4/ipv6 can also be used as a form of backup ips or the primary ips for their respective versions. Similar to A & AAAA records.
-			ip: ['::1', '127.0.0.1'],
+			// ip: ['::1', '127.0.0.1'],
+			ip: '::1',
 			// This allows for IPv4 and IPv6 to use specific details that is more fitting for that endpoint.
 			// This is not intended to be used when IPv6 becomes the standard.
 			// ipv4: '127.0.0.1',
@@ -103,8 +104,28 @@ const domainProfile = await createProfile({
 			// Max sizes of the body/data/payload for upload and download
 			// maxUploadSize: 1024,
 			// maxDownloadSize: 1024,
-			// Shows the browser how to display the domain name for humans correctly
-			domainTextEncoding: 'utf8',
+			// crypto: {
+			// type: 'viat',
+			// enable viat puzzles as a form of congestion control
+			// puzzles: true,
+			// When publicKey is set to true it will use the public key in the certificate as the main Viat wallet for the domain. If a string is provided then it would be the main wallet for the domain.
+			// publicKey: true
+			// },
+			// This allows a browser or connection to use the realtime mode for the UDSP connection
+			// Permits it to act similar to a web transport and or a websocket connection
+			// realtime: true,
+			// blocked methods mean methods the server doesn't permit
+			// blockedMethods: ['delete'],
+			// allowedMethods: ['get', 'connect', 'file', 'stream', 'close'],
+			// deafult compression of payload data only
+			// compression: true,
+			// packetCompression: true,
+			// headerCompression: true,
+			// footerCompression: true,
+			// messageCompression: true,
+			// autoLogin: true,
+		},
+		whois: {
 			entity: {
 				name: 'Universal Web',
 			},
@@ -115,27 +136,6 @@ const domainProfile = await createProfile({
 				town: 'UW Township',
 				county: 'UW County',
 			},
-			crypto: {
-				type: 'viat',
-				// enable viat puzzles as a form of congestion control
-				puzzles: true,
-				// When publicKey is set to true it will use the public key in the certificate as the main Viat wallet for the domain. If a string is provided then it would be the main wallet for the domain.
-				publicKey: true
-			},
-			// This allows a browser or connection to use the realtime mode for the UDSP connection
-			// Permits it to act similar to a web transport and or a websocket connection
-			realtime: true,
-			// blocked methods mean methods the server doesn't permit
-			// blockedMethods: ['delete'],
-			allowedMethods: ['get', 'connect', 'file', 'stream', 'close'],
-			// deafult compression of payload data only
-			compression: true,
-			// packetCompression: true,
-			// headerCompression: true,
-			// footerCompression: true,
-			// pathCompression: true,
-			// parameterCompression: true,
-			autoLogin: true,
 		},
 		master: {},
 	},

scripts/simulateClient.js

@@ -1,6 +1,6 @@
 console.clear();
 const logFunction = console.log;
-console.log = () => {};
+// console.log = () => {};
 console.log('STARTING CLIENT');
 console.time('Full');
 import { currentPath } from '@universalweb/acid';
@@ -24,7 +24,7 @@ const fileRequest = await uwClient.request('get', 'index.html');
 // console.log(fileRequest);
 console.time('FileRequest');
 const response = await fileRequest.send();
-console.log = logFunction;
+// console.log = logFunction;
 console.timeEnd('FileRequest');
 console.timeEnd('Full');
 console.log('fileRequest', fileRequest);

udsp/base.js

@@ -42,7 +42,7 @@ export class UDSP {
 			}
 		} else {
 			const packetInitialOverhead = 2;
-			const connectionIdSize = (this.isClient) ? this.clientConnectionIdSize : this.connectionIdSize;
+			const connectionIdSize = this.connectionIdSize;
 			this.encryptPacketOverhead = this.encryptOverhead;
 			this.packetOverhead = packetInitialOverhead + this.encryptPacketOverhead + connectionIdSize;
 			this.maxPacketPayloadSize = this.maxPacketSize - this.packetOverhead;

udsp/client/index.js

@@ -38,7 +38,7 @@ import { watch } from '#watch';
 // Client specific imports to extend class
 import { emit } from '../requestMethods/emit.js';
 import { uwRequest } from '#udsp/requestMethods/request';
-import { processMessage } from './processMessage.js';
+import { processFrame } from './processFrame.js';
 import { onPacket } from './onPacket.js';
 import { onListening } from './listening.js';
 import { keychainGet } from '#keychain';
@@ -79,7 +79,7 @@ export class Client extends UDSP {
 			ipVersion
 		} = this;
 		if (isString(destinationCertificate)) {
-			// console.log('Loading Destination Certificate', destinationCertificate);
+			console.log('Loading Destination Certificate', destinationCertificate);
 			const certificate = await getCertificate(destinationCertificate);
 			assign(destination, certificate);
 		} else {
@@ -112,6 +112,9 @@ export class Client extends UDSP {
 		if (port) {
 			destination.port = port;
 		}
+		if (this.destination.clientConnectionIdSize) {
+			this.connectionIdSize = this.destination.clientConnectionIdSize;
+		}
 		// console.log('Destination', destination.cryptography);
 	}
 	async getKeychainSave(keychain) {
@@ -166,11 +169,11 @@ export class Client extends UDSP {
 		if (!this.encryptionKeypair) {
 			this.encryptionKeypair = this.keypair;
 		}
-		await this.setSessionKeys();
 		const convertSignKeypairToEncryptionKeypair = processPublicKey(this.destination);
 		if (convertSignKeypairToEncryptionKeypair) {
 			this.destination.encryptionKeypair = convertSignKeypairToEncryptionKeypair;
 		}
+		await this.setSessionKeys();
 		if (encryptConnectionId) {
 			const {
 				server: encryptServerCid,
@@ -243,7 +246,7 @@ export class Client extends UDSP {
 		this.idSize = this.id.length;
 		success(`clientId:`, this.idString);
 		await this.setDestination();
-		await this.setCertificate();
+		await this.setProfile();
 		console.log('ipVersion', this.ipVersion);
 		console.log('destination', this.destination);
 		await this.configCryptography();
@@ -294,6 +297,7 @@ export class Client extends UDSP {
 		}
 	}
 	async setSessionKeys(generatedKeys) {
+		console.log(this.destination.encryptionKeypair);
 		this.sessionKeys = generatedKeys || this.publicKeyCryptography.clientSessionKeys(this.encryptionKeypair, this.destination.encryptionKeypair);
 		if (this.sessionKeys) {
 			success(`Created Shared Keys`);
@@ -315,21 +319,21 @@ export class Client extends UDSP {
 			this.handshakeSet = true;
 		}
 	}
-	async intro(message) {
-		console.log('Got server Intro', message);
-		if (!message || !isArray(message)) {
-			this.close('No intro message');
+	async intro(frame, header) {
+		if (!frame || !isArray(frame)) {
+			this.close('No intro message', frame);
 			return;
 		}
-		const [streamid_undefined, rpc, serverConnectionId, reKey, serverRandomToken, certSize] = message;
+		console.log('Got server Intro', frame);
+		const [streamid_undefined, rpc, serverConnectionId, reKey, serverRandomToken, certSize, dataSize] = frame;
 		this.destination.id = serverConnectionId;
 		this.destination.idSize = serverConnectionId.length;
 		this.newKeypair = reKey;
 		await this.setNewDestinationKeys();
 		console.log('New Server Connection ID', toBase64(serverConnectionId));
 		if (serverRandomToken) {
 			this.serverRandomToken = serverRandomToken;
-			console.log('Server Random Token', toBase64(serverConnectionId));
+			console.log('Server Random Token', toBase64(serverRandomToken));
 		}
 		if (certSize) {
 			this.certSize = certSize;
@@ -342,7 +346,8 @@ export class Client extends UDSP {
 		this.state = 2;
 		this.readyState = 1;
 		// Resolve the handshake promise
-		this.handshakeCompleted(message);
+		console.log(this.handshakeCompleted);
+		this.handshakeCompleted();
 	}
 	setPublicKeyHeader(header = []) {
 		const key = this.encryptionKeypair.publicKey;
@@ -383,9 +388,11 @@ export class Client extends UDSP {
 	}
 	ensureHandshake() {
 		if (this.connected === true) {
+			console.log('ALREADY CONNECTED');
 			return true;
 		} else if (!this.handshakeCompleted) {
 			this.awaitHandshake = promise((accept) => {
+				console.log('HANDSHAKE AWAITING');
 				this.handshakeCompleted = accept;
 			});
 			this.sendIntro();
@@ -396,32 +403,17 @@ export class Client extends UDSP {
 		console.log(`client.send to Server`, this.destination.ip, this.destination.port);
 		return sendPacket(message, this, this.socket, this.destination, headers, footer);
 	}
-	proccessProtocolPacket(message) {
-		const {
-			i: intro,
-			certIndex,
-			handshake,
-			state
-		} = message;
-		console.log('Processing Protocol Packet', message);
-		if (intro) {
-			if (certIndex) {
-				this.proccessCertificateChunk(message);
-			} else {
-				this.intro(message);
-			}
-		} else if (handshake) {
-			this.handshaked(message);
-		} else if (state) {
-			if (state === 3) {
-				this.close(message);
-			}
+	proccessProtocolPacket(frame, header) {
+		const rpc = frame[1];
+		console.log('Processing Protocol Packet', frame);
+		if (rpc === 0) {
+			this.intro(frame, header);
 		}
 	}
 	request = uwRequest;
 	fetch = fetchRequest;
 	post = post;
-	processMessage = processMessage;
+	processFrame = processFrame;
 	emit = emit;
 	onListening = onListening;
 	onPacket = onPacket;

udsp/client/onPacket.js

@@ -1,11 +1,11 @@
 import {
 	success, failed, imported, msgSent, info, msgReceived
 } from '#logs';
-import { decode } from '#utilities/serialize';
+import { decode, encode } from '#utilities/serialize';
 import { decrypt, createSessionKey } from '#crypto';
 import { decodePacket, decodePacketHeaders } from '#udsp/encoding/decodePacket';
-import { processMessage } from './processMessage.js';
-import { hasValue } from '@universalweb/acid';
+import { processFrame } from './processFrame.js';
+import { hasValue, isArray } from '@universalweb/acid';
 export async function onPacket(packet) {
 	msgReceived('Packet Received');
 	const config = {
@@ -27,10 +27,13 @@ export async function onPacket(packet) {
 		message,
 		footer,
 	} = config.packetDecoded;
-	if (hasValue(message?.id)) {
-		processMessage(config.packetDecoded, this);
-	} else {
-		this.proccessProtocolPacket(message, header, footer);
+	if (isArray(message)) {
+		const rpc = message[1];
+		if (rpc === 0) {
+			this.proccessProtocolPacket(message, header);
+		} else {
+			processFrame(message, header, this);
+		}
 	}
 }