Giggle Forest (project.VoiceSpace) by Team - under5

Development Goals

As we go through the COVID-19, most of the 'meeting places' are rapidly moving to various virtual conference spaces started from Zoom. As such, there are many side effects, A typical example is Zoom Fatigue, which causes a lot of fatigue in virtual conferences than usual conversations. We think the main causes of that are

1. video
2. Rigid video conferencing space Accordingly, we started to development this service with the purpose of creating a service that provides a video-free, realistic conversation, and a soft and enjoyable space.

---

Introduction Video (with Korean language)

Demo Website Link

Under construction of demo website.

• url : https://giggleforest.com
• you can try it right here

---

Development and deployment environment

Part	Environment	Remark(Version)
FrontEnd	React	17.0.2
BackEnd	NestJS	7.6.15
WebServer	Nginx	1.14.2
Publishing Server Hardware	AWS	EC2

---

Main library

Name	Remark
WebRTC	Seamless voice and data transmission between users
WebGL(PixiJS)	Outputs animation instead of video, using the client's graphics processing hardware
WebAudioAPI	Voice analysis and control for realizing realistic conversations without video
Jest	Unit and Integration Test of implemented components and classes

---

System diagram

Overall system configuration

• Peer-to-peer connection using WebRTC is the most important part.
• Audio streams are exchanged through this P2P connection, and avatar-related data is also continuously transmitted to each other.
• Therefore, when a user connects for the first time, the P2P connection is established using the back signaling server.
• As the connection is completed, the transmitted audio stream is continuously played, and animation is drawn every frame using WebGL through the transmitted avatar information.

Multiple peer connection architecture

• So we aim for a project that anyone can service on their own server, that a number of P2P connections were established in Full Mesh in order to minimize the role of the back server.
• If we launch this service, the architecture may be restructured using SFU.

Page Infomation

1. Home Page

Component Path	/front/src/pages/homePage/Home.tsx
URL	/
Role	Route to SettingComponent (/setting) with random or specific(input) roomId
Remark	자세히 알아보세요 is connected with this github page

---

2. Setting Page

Component Path	/front/src/pages/settingPage/Setting.tsx
URL	/setting
Role	Select mic, speaker device and avatar information (nickname and avatarImage) and route to SpaceComponent with selected values
Remark	Only Chrome browser is fully supported

---

3. Space Page

Component Path	/front/src/pages/spacePage/Space.tsx
URL	/space
Role	- Draw map and avatars using pixiUtils - Establish P2P connection and transfer avatar data using Peer, PeerManager - Analyze voice for lip sync using AudioAnalyer - Control all of settings (with Navigation Component
Remark	- If you want to customize world map, see World.ts and world1.json - If you want to add some special stuff (like Youtube sign), see Stuff.ts - If you want to Improve lip sync logic, see AudioAnalyer - If you want to develop your own signaling server, see RTCSignalingHelper.ts - If you want to add stun/turn server, see IceServerList.ts

---

Main function

• Smooth animation using WebGL in a outstanding level (reducing eye fatigue)
• Real-time voice chat with very low latency using WebRTC (reducing fatigue from delay)
• The more a distance between avatars increases, the more voice volume also decreases (like realistic meeting)
• A function that analyzes the voice and changes the shape of the avatar's mouth to match the pronunciation (like realistic meeting)
• The size of the avatar's face changes according to the volume of the voice. (realistic voice visualization)
• 

you can try it right here

---

Main features being added (Advancement)

• map customization

---

Start project

in back folder

npm install
npm run start:dev

in front folder

npm install
npm run start

• You can change the backend address in .env.development in the front folder.

---

Deploy the project

Front

1. npm run build (in front folder) The value of (.env.production) applies.
2. Serve the build folder output from the above script as root (our team used nginx)

Back

1. npm run build (in back folder) (In this case, if you want to set it to https, please change the contents of main.ts (see comments))
2. Execute main.js in the dist folder as the output of the above script as NodeJS.

---

Document

• Information on various components, classes, and functions can be found here.

---

Test

In front folder

• unit test

npm run test

• unit test with coverage report

npm run testCover

• The current master's coverage report can be viewed here

---

License

• MIT

---

Team Information

Name	Email	Role	Major Part	Minor Part	Tech Stack
kilee	[email protected]	Leader	Design, project deployment and management	Front	AWS, Github Action, CI/CD, React, Typescript
honlee	[email protected]	member	Front	Back	AWS, React, NestJS, Typescript
hyeonkim	[email protected]	member	Back	Front	React, NestJS, Typescript
mijeong	[email protected]	member	Front	Back	React, NestJS, Typescript