first test

km0 1 year ago
parent 3d08feaefb
commit e5e908e65a

4
.gitignore vendored

@ -1 +1,3 @@
node_modules/
node_modules/
.env
package-lock.json

@ -1,8 +1,7 @@
# DRW
# Teletap
![Drawing a bird](cover.jpg)
A small playground for networked sound experiments with Web Audio API.
A small app for collecting drawings in real time. Runs on a small express server that connects sources (where to draw) and destinations (where to display) via websockets.
## Setup
@ -26,181 +25,8 @@ or in alternative
`npm start`
Then open your browser to `localhost:3000` and there you can draw.
Then open your browser to `localhost:3000` and there you can tap on the screen to generate sounds..
If you open another tab and navigate to `localhost:3000/destination`, there you will receive the drawings.
If you open another tab and navigate to `localhost:3000/destination`, there you will receive the sounds. Note that you will not hear anything until you press on the `Join Audio` button. This is because for security policies, browsers don't play any sound before a user's interaction.
This destination page is just an example! The app is meant to be open-end, meaning that the destination of the drawing is up to you! (ah ah! less work for us). Originally it was coupled with [vvvv](https://visualprogramming.net/), but it can be implemented with any platform supporting the websocket protocol (the browser, pure data, max, touch designer, p5js, etc).
## How does it work
![A websocket connection is like ping pong](img/pingpong.jpg)
This app works like the game of ping pong. Two (or more) atlethic players connect to a server to exchange messages through the fast ball of [Websocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API).
There are four main elements in this app:
**Source**
A source is a client that connects to the application in order to send drawings.
To connect as a source simply connect to the server.
This is what happens when you visit the [drawing page](https://hub.xpub.nl/soupboat/drw/).
For an example, look at the script section in the `views/index.html` file in the folder.
**Destination**
A destination is a client that connects to the application in order to receive drawings.
To connect as a destination, greet the server with an hello message:
`{"type": "hello"}`
This is what happens when you visit the [destination page](https://hub.xpub.nl/soupboat/drw/destination). For an example of the code, look at the script section in the `view/destination.html` section. Right after opening the websocket connection, the script will send a greeting to the server.
**Websocket Messages**
Websockets are a type of connection especially suited to real time application.
Here they are small `JSON` messages of different types. Imagine it as small Javascript objects or Python dictionaries
```
{
"type": "test",
"data": "hello this is a test message!"
}
```
Here they always come with a `type` property, that is used in both server and clients to trigger specific functions depending on it.
At the moment the app logic is built on these messages:
- **hello**
register the client as a destination
_example:_
`{"type": "hello"}`
- **drawings**
message that contains [SVG path](https://developer.mozilla.org/en-US/docs/Web/SVG/Tutorial/Paths) of a drawing
_example:_
`{"type": "drawings", "paths": "M 10 10 H 90 V 90 H 10 L 10 10"}`
maybe should be renamed just drawing
- **theme**
message that suggests to the player a theme to draw about
_example:_
`{"type": "theme", "theme": "animals"}`
**Server**
The server is the table that grants to the connected clients to exchange messages. It mainly takes care of keeping track of who is connected, and to send the right message to the right client. For an insight of how does it work look at the comments in the `server.js` file!
## Going online
Eventually you want to put online your drawing app.
To be able to use this app on the [Soupboat](hub.xpub.nl/soupboat) (or other servers connected in the [hub.xpub.nl](hub.xpub.nl) ecosystem) some additional configurations are needed.
Note that the following details are tailored to the particular case of our server. Other instances could require different setups.
This is one possible workflow.
Clone the repo and install the requirements as you would do locally.
```
git clone https://git.xpub.nl/kamo/drw
cd drw
npm install
```
### Environmental variables
There are a couple of environmental variables to set: one refers to the port where to mount the application, the other is related to the prefix to add to the application urls.
```
nano .env
```
Will create a new `.env` file where to add the values for this specific environment.
In the case of the soupboat, for example:
```
PORT=3000
PREFIX=/soupboat/drw/
```
Save and exit.
The port is where Express will mount the application. This is by default set to 3000, but in this case we need to pick a port not already in use.
When deciding which port to use, check your NGINX configurations file (see next section), or simply test if the port you want is already in use.
`sudo lsof -i:3000`
For example, will print the process currently using the port 3000. If nothing is printed out, then the port is available.
Read more about it here: [Check if port is in use Linux](https://www.cyberciti.biz/faq/unix-linux-check-if-port-is-in-use-command/)
The prefix variable is a way to deal with the _hub.xpub.nl_ ecosystem. Here our base url is `hub.xpub.nl`. Notice that is missing the `/soupboat/drw/` part.
The deal of the prefix is to leave out from the code these parts of the address, that otherwise should be repeated in every url and navigation element of the app.
This also make the code a bit more portable, meaning that you can test it locally and online without messing around with the urls in the code.
The app is written in order to provide some default values if an `.env` file is not found, and that's why it works locally even without specifying any environmental variables.
### NGINX Configuration
To make it works behind a reverse-proxy open the NGINX configuration file
```
sudo nano /etc/nginx/sites-available/default
```
and inside the server section add a new location:
```
server {
#note that your configurations may differ!
listen 80 default_server;
listen [::]:80 default_server;
root /var/www/html
# ADD FROM HERE
location /drw/ {
proxy_pass http://localhost:3000/soupboat/drw/;
include proxy_params;
proxy_set_header Host $http_host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
# TO HERE
}
```
The lines that you should edit according your configurations are:
- `location /drw/`
The name of the location _/drw/_ is up to you, and it's the address where the app will be served. In this case will be _hub.xpub.nl/soupboat/drw/_.
- `proxy_pass http://localhost:3000/soupboat/drw/;`
The very same name, as well as eventual prefixes, need to be specified in the line of _proxy pass_.
The port, in this example set to _3000_, it's the port where Express is mounting the application. By default is 3000, but you can edit it according to the configurations of the express server.
The three `proxy_set_header` Host, Upgrade and Connection are necessary to make the Websocket connection work.
Once you add these info save and exit.
To check that the NGINX configuration file is ok run
`sudo nginx -t`
If it prints that everything is fine, reload nginx to apply the configurations.
If there are errors instead reopen the configurations file and fix them!
**Watch out**: reloading nginx when the configurations are broken means disaster. Always run the test before reloading!
Then you can start the app as you would do locally.
`node server.js`
This app uses the same architecture of [DRw](https://git.xpub.nl/kamo/drw). Find more info there!

@ -0,0 +1,131 @@
const audioContext = new AudioContext();
const tap = document.querySelector('.container')
const address = document.querySelector('#address').innerHTML
const socket = new ReconnectingWebSocket(
location.origin.replace(/^http/, "ws") + address
);
socket.onopen = (event) => {
socket.send(JSON.stringify({ type: "hello" }));
console.log("Connected as destination!");
};
let noiseDuration = 0.05
const playNoise = (bandHz = 1000, time = 0) => {
const bufferSize = audioContext.sampleRate * noiseDuration
// Create an empty buffer
const noiseBuffer = new AudioBuffer({
length: bufferSize,
sampleRate: audioContext.sampleRate,
})
// Fill the buffer with noise
const data = noiseBuffer.getChannelData(0);
for (let i = 0; i < bufferSize; i++) {
data[i] = Math.random() * 2 - 1;
}
// Create a buffer source from data
const noise = new AudioBufferSourceNode(audioContext, {
buffer: noiseBuffer,
})
// Filter the output
const bandpass = new BiquadFilterNode(audioContext, {
type: "bandpass",
frequency: bandHz
})
noise.connect(bandpass).connect(audioContext.destination);
noise.start(time);
}
const playPulse = (freq=440, lfoFreq=30, duration=1, time=0) => {
const osc = new OscillatorNode(audioContext, {
type: "square",
frequency: freq
})
const amp = new GainNode(audioContext, {
value: 0
})
const lfo = new OscillatorNode(audioContext, {
type: "sine",
frequency: lfoFreq
})
lfo.connect(amp.gain)
osc.connect(amp).connect(audioContext.destination)
lfo.start()
osc.start(time)
// osc.stop(time + duration)
}
const spawnGradient = (x, y) => {
const gradient = document.createElement('div')
gradient.classList.add('gradient')
gradient.style.translate = `${x}px ${y}px`
gradient.style.scale = 0
let red = x / tap.clientWidth * 255
let green = y / tap.clientHeight * 255
let blue = 0
gradient.style.background = `radial-gradient(circle, rgba(${red},${green},${blue},1) 0%, rgba(${red},${green},${blue},0) 25%)`
tap.appendChild(gradient)
grow(gradient)
}
const grow = (el) => {
let scale = Number(el.style.scale) || 0
el.style.scale = scale + 0.1
requestAnimationFrame(()=> grow(el))
}
const emit = (x, y) => {
playPulse(x, y * 0.01)
spawnGradient(x, y)
}
const join = document.querySelector('#join')
join.addEventListener('click', ()=>{
playNoise()
})
socket.onmessage = (event) => {
let message;
try {
message = JSON.parse(event.data);
} catch (e) {}
console.log("received a message! ", message)
if (message?.type == 'pulse'){
emit(message.x, message.y)
}
};

@ -1,33 +1,21 @@
html,
body {
font-family: sans-serif;
background-color: dodgerblue;
.container {
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 100vh;
}
#svgElement {
background-color: white;
.container.destination {
pointer-events: none;
}
.destination #svgElement {
background: none;
}
button {
display: block;
background-color: white;
border: 2px solid currentColor;
padding: 8px 24px;
border-radius: 24px;
font-size: 24px;
margin-top: 16px;
}
.hidden {
display: none;
}
#theme {
font-weight: bold;
-webkit-text-stroke: 1px black;
-webkit-text-fill-color: white;
.gradient {
position: absolute;
top: 0;
left: 0;
width: 50px;
height: 50px;
display: inline-block;
pointer-events: none;
}

@ -0,0 +1,100 @@
const tap = document.querySelector('.container')
const address = document.querySelector('#address').innerHTML
const socket = new ReconnectingWebSocket(location.origin.replace(/^http/, "ws") + address);
const audioContext = new AudioContext();
let noiseDuration = 0.05
const playNoise = (bandHz = 1000, time = 0) => {
const bufferSize = audioContext.sampleRate * noiseDuration
// Create an empty buffer
const noiseBuffer = new AudioBuffer({
length: bufferSize,
sampleRate: audioContext.sampleRate,
})
// Fill the buffer with noise
const data = noiseBuffer.getChannelData(0);
for (let i = 0; i < bufferSize; i++) {
data[i] = Math.random() * 2 - 1;
}
// Create a buffer source from data
const noise = new AudioBufferSourceNode(audioContext, {
buffer: noiseBuffer,
})
// Filter the output
const bandpass = new BiquadFilterNode(audioContext, {
type: "bandpass",
frequency: bandHz
})
noise.connect(bandpass).connect(audioContext.destination);
noise.start(time);
}
const playPulse = (freq=440, lfoFreq=30, duration=1, time=0) => {
const osc = new OscillatorNode(audioContext, {
type: "square",
frequency: freq
})
const amp = new GainNode(audioContext, {
value: 0
})
const lfo = new OscillatorNode(audioContext, {
type: "sine",
frequency: lfoFreq
})
lfo.connect(amp.gain)
osc.connect(amp).connect(audioContext.destination)
lfo.start()
osc.start(time)
// osc.stop(time + duration)
}
const spawnGradient = (x, y) => {
const gradient = document.createElement('div')
gradient.classList.add('gradient')
gradient.style.translate = `${x}px ${y}px`
gradient.style.scale = 0
let red = x / tap.clientWidth * 255
let green = y / tap.clientHeight * 255
let blue = 0
gradient.style.background = `radial-gradient(circle, rgba(${red},${green},${blue},1) 0%, rgba(${red},${green},${blue},0) 25%)`
tap.appendChild(gradient)
grow(gradient)
}
const grow = (el) => {
let scale = Number(el.style.scale) || 0
el.style.scale = scale + 0.1
requestAnimationFrame(()=> grow(el))
}
tap.addEventListener("click", (e) => {
playPulse(e.clientX, e.clientY * 0.01)
spawnGradient(e.clientX, e.clientY)
console.log('sending pulse')
socket.send(JSON.stringify({type: 'pulse', x: e.clientX, y: e.clientY }))
})

@ -19,7 +19,7 @@ const PUBLIC = process.env.PUBLIC || "";
const router = express.Router();
const routes = (app) => {
app.get("/", (req, res) => {
res.render("index", {
res.render("index", {
address: PREFIX,
});
});
@ -27,11 +27,6 @@ const routes = (app) => {
res.render("destination", {
address: PREFIX,
});
});
app.get("/wander", (req, res)=>{
res.render("wander", {
address: PREFIX,
})
});
app.get("/*", (req, res) => {
res.sendFile(req.url, { root: "public" });
@ -63,8 +58,7 @@ var theme = "";
const messageProcessor = {
default: (ws, msg) => unknownMsg(msg),
hello: (ws, msg) => registerDest(ws, msg),
drawings: (ws, msg) => toDest(msg),
theme: (ws, msg) => ((theme = msg.theme), broadcast(msg)),
pulse: (ws, msg) => toDest(msg)
};
// Message processor functions
@ -98,6 +92,7 @@ const toDest = (msg) => {
// Send a message to all the connected Users
const broadcast = (msg) => {
let message = JSON.stringify(msg);
console.log('broadcasting: ', message)
for (const user of USERS.values()) {
if (user?.readyState === WebSocket.OPEN && !DESTINATIONS.has(user)) {
user.send(message);

Binary file not shown.

@ -6,58 +6,13 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Draw draw draw</title>
<script src="wss.js"></script>
<script src="audioDestination.js" defer></script>
<link rel="stylesheet" href="style.css" />
</head>
<body>
<h1>Display</h1>
<div id="svg-container" class="destination">
<svg
class="hidden"
xmlns="http://www.w3.org/2000/svg"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="1.1"
id="svgElement"
x="0px"
y="0px"
width="200px"
height="200px"
viewBox="0 0 500 500"
enable-background="new 0 0 100 100"
preserveAspectRatio="none"
xml:space="preserve"
></svg>
</div>
<div class="container destination"></div>
<span id="address">{{address}}</span>
<button id="join">Join audio</button>
<script>
const svgModel = document.querySelector("#svgElement");
const container = document.querySelector("#svg-container");
const socket = new ReconnectingWebSocket(
location.origin.replace(/^http/, "ws") + "{{address}}"
);
socket.onopen = (event) => {
socket.send(JSON.stringify({ type: "hello" }));
console.log("Connected as destination!");
};
socket.onmessage = (event) => {
let message;
try {
message = JSON.parse(event.data);
} catch (e) {}
if (message?.type == "drawings") {
let svg = svgModel.cloneNode();
svg.classList.remove("hidden");
let path = document.createElementNS("http://www.w3.org/2000/svg", "path");
path.setAttribute("d", message.paths);
path.setAttribute("fill", "none");
path.setAttribute("stroke", "currentColor");
path.setAttribute("stroke-width", 8);
svg.appendChild(path);
container.appendChild(svg);
}
};
</script>
</body>
</html>

@ -6,66 +6,14 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Draw draw draw</title>
<script src="wss.js"></script>
<script src="draw.js" defer></script>
<link rel="stylesheet" href="style.css" />
<script src="teletap.js" defer></script>
<link rel="stylesheet" href="style.css" />
</head>
<body>
<h1>Draw <span id="theme"></span></h1>
<div id="svg-container">
<svg
xmlns="http://www.w3.org/2000/svg"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="1.1"
id="svgElement"
x="0px"
y="0px"
width="500px"
height="500px"
viewBox="0 0 500 500"
enable-background="new 0 0 500 500"
xml:space="preserve"
></svg>
<button id="submit">Send</button>
<button id="erase">Cancel</button>
</div>
<script>
const theme = document.querySelector("#theme");
const erase = document.querySelector("#erase");
const deletePath = () => {
for (const path of document.querySelectorAll("#svgElement path")) {
path.remove();
}
};
<span id="address">{{address}}</span>
<div class="container"></div>
erase.addEventListener("click", () => deletePath());
const socket = new ReconnectingWebSocket(
location.origin.replace(/^http/, "ws") + "{{address}}"
);
socket.onmessage = (event) => {
let message;
try {
message = JSON.parse(event.data);
} catch (e) {}
if (message?.type == "theme") {
theme.innerHTML = message.theme;
}
};
document.querySelector("#submit").addEventListener("click", (event) => {
const paths = Array.from(document.querySelectorAll("#svgElement path")).reduce(
(accumulator, current) => {
return (accumulator += current.getAttribute("d"));
},
""
);
if (paths == "") return;
socket.send(JSON.stringify({ type: "drawings", paths: paths }));
deletePath();
});
</script>
</body>
</html>

Loading…
Cancel
Save