In this article, I’m going to take you on an adventure with me. Let’s implement the real-time collaborative text editor using tiptap and our own socket.io server.
Vocabulary
Before going further, let’s check some definitions of concepts we mention in this article:
tiptap
tiptap is a renderless rich-text editor for Vue.js. Renderless means the developer has full control over markup and styling. It is based on ProseMirror, which is a toolkit for building rich-text editors that are already in use at many well-known companies such as Atlassian or New York Times.
Tiptap provides a simple but limited example of collaborative editing. For starters, it allows to only edit a single document. Naept handles a lot of elements containing rich-text sections. Up to now, updating those elements was made using dialog boxes, and sending update requests to the back-end. It works well, but it’s not user-friendly. I want to enhance the collaborative aspect of naept by providing collaborative text edition for every rich-text area of the UI. For that, we have to upgrade both client-side and server-side code from the tiptap example in order to handle many documents.
Making the tiptap collaborative editor example work locally
Making the client-side code work locally is pretty easy. Steps are described on the tiptap GitHub page:
# install dependencies
yarn install
# serve examples at localhost:3000
yarn startBut this example still connects to the example socket server they hosted on glitch.com. We want it do connect to a server running on our local machine.
In order to make a socket.io server, we first need to install NodeJS. This is a pretty straightforward step so I’ll let you handle it by yourself.
Now, on our computer, we create a directory for our server project, and copy the files from the tiptap example hosted on glitch: server.js, schema.js, package.json and db_locked.json are to be kept untouched, db_steps.json can contain an empty array [], and db.json may be missing.
Before going further, we just make a quick change in the server.js file. As the 3000 port is already in use by the client, we change the http.listen(3000) instruction on line 9 for http.listen(6000). Or any other port you want if 6000 is not free on your computer.
We can now run the socket server:
# In the server project directory
npm startNow we can change the address of the socket server in the tiptap example file examples/Components/Collaboration/index.vue, line 88:
this.socket = io('wss://tiptap-sockets.glitch.me')becomes:
this.socket = io('http://localhost:6000')We may refresh the http://localhost:3000/collaboration page in our browser and now see that we are connected to our local server:
Creating multiple documents using namespaces
What I want to do now is create some kind of channels so that tiptap can be used to edit multiple documents.
There is two ways of “splitting” a web socket with socket.io: namespaces and rooms. Actually, a socket can be divided into namespaces, and namespaces can be divided into rooms.
The clients can choose a namespace to connect to, but have no direct hand over which room they will join. Namespaces can be protected by user authorization, whereas it’s not natively handled for rooms.
As my application is intended to handle many documents grouped in projects, I will – and I think it’s a nice opportunity to explore both in this article – use a room for each document, and a namespace for each project.
In this particular case, the server is not aware of all the namespaces. They are given by the client when it connects. As specified in the documentation, we are going to use a regular expression to handle namespaces.
io.on('connection', socket => {...})becomes:
const namespaces = io.of(/^\/[a-zA-Z0-9_\/-]+$/)
namespaces.on('connection', socket => {
const namespace = socket.nsp;
[...]
})The regular expression allows to have slashes. This will give the opportunity to create subdomains. Although it won’t mean anything for socket.io, it’ll help to structure and organize the different documents.
The current namespace is retrievable via socket.nsp. We store it in a constant named namespace and then every io.emit() or io.sockets.emit() must be replaced by namespace.emit().
At his point, our socket server handles connections from clients using URLs like http://localhost:6002/doc1 or even http://localhost:6002/projectA/doc3, but as we did not make any change on the server’s files management, we won’t see any difference in using a different URL. Indeed, the location for the db.json, db_locked.json and db_steps.json files is still defined by constants while we would like to define it dynamically using the current namespace’s name.
So to every function writing or reading a file we now add a parameter to specify the directory in which the file should be stored.
Here is what the server.js looks like with all those modifications:
import fs from 'fs'
import { Step } from 'prosemirror-transform'
import schema from './schema.js'
// setup socket server
const app = require('express')()
const http = require('http').Server(app)
const io = require('socket.io')(http)
http.listen(6002)
// options
const simulateSlowServerDelay = 0 // milliseconds
const dbPath = './db'
const docPath = '/db.json'
const lockedPath = '/db_locked.json'
const stepsPath = '/db_steps.json'
const maxStoredSteps = 1000
const defaultData = {
"version": 0,
"doc": { "type": "doc", "content": [{ "type": "paragraph", "content":[{ "type": "text", "text": "Let's start collaborating. Yeah!" }] }] }
}
const sleep = (ms) => (new Promise(resolve => setTimeout(resolve, ms)));
function initProjectDir(namespaceDir) {
if (!fs.existsSync(dbPath + namespaceDir)){
fs.mkdirSync(dbPath + namespaceDir, { recursive: true })
}
}
function storeDoc(data, namespaceDir) {
fs.writeFileSync(dbPath + namespaceDir + docPath, JSON.stringify(data, null, 2))
}
function storeSteps({steps, version}, namespaceDir) {
let limitedOldData = []
try {
const oldData = JSON.parse(fs.readFileSync(dbPath + namespaceDir + stepsPath, 'utf8'))
limitedOldData = oldData.slice(Math.max(oldData.length - maxStoredSteps))
} catch(e) {
}
const newData = [
...limitedOldData,
...steps.map((step, index) => {
return {
step: JSON.parse(JSON.stringify(step)),
version: version + index + 1,
clientID: step.clientID,
}
})
]
fs.writeFileSync(dbPath + namespaceDir + stepsPath, JSON.stringify(newData))
}
function storeLocked(locked, namespaceDir) {
fs.writeFileSync(dbPath + namespaceDir + lockedPath, locked.toString())
}
function getDoc(namespaceDir) {
try {
return JSON.parse(fs.readFileSync(dbPath + namespaceDir + docPath, 'utf8'))
} catch(e) {
return defaultData
}
}
function getLocked(namespaceDir) {
try {
return JSON.parse(fs.readFileSync(dbPath + namespaceDir + lockedPath, 'utf8'))
} catch(e) {
return false
}
}
function getSteps(version, namespaceDir) {
try {
const steps = JSON.parse(fs.readFileSync(dbPath + namespaceDir + stepsPath, 'utf8'))
return steps.filter(step => step.version > version)
} catch(e) {
return []
}
}
const namespaces = io.of(/^\/[a-zA-Z0-9_\/-]+$/)
namespaces.on('connection', socket => {
const namespace = socket.nsp;
const namespaceDir = namespace.name
initProjectDir(namespaceDir)
socket.on('update', async ({ version, clientID, steps }) => {
// we need to check if there is another update processed
// so we store a "locked" state
const locked = getLocked(namespaceDir)
if (locked) {
// we will do nothing and wait for another client update
return
}
storeLocked(true, namespaceDir)
const storedData = getDoc(namespaceDir)
await sleep(simulateSlowServerDelay)
// version mismatch: the stored version is newer
// so we send all steps of this version back to the user
if (storedData.version !== version) {
namespace.emit('update', {
version,
steps: getSteps(version, namespaceDir),
})
storeLocked(false, namespaceDir)
return
}
let doc = schema.nodeFromJSON(storedData.doc)
await sleep(simulateSlowServerDelay)
let newSteps = steps.map(step => {
const newStep = Step.fromJSON(schema, step)
newStep.clientID = clientID
// apply step to document
let result = newStep.apply(doc)
doc = result.doc
return newStep
})
await sleep(simulateSlowServerDelay)
// calculating a new version number is easy
const newVersion = version + newSteps.length
// store data
storeSteps({ version, steps: newSteps }, namespaceDir)
storeDoc({ version: newVersion, doc }, namespaceDir)
await sleep(simulateSlowServerDelay)
// send update to everyone (me and others)
namespace.emit('update', {
version: newVersion,
steps: getSteps(version, namespaceDir),
})
storeLocked(false, namespaceDir)
})
// send latest document
namespace.emit('init', getDoc(namespaceDir))
// send client count
namespace.emit('getCount', io.engine.clientsCount)
socket.on('disconnect', () => {
namespace.emit('getCount', io.engine.clientsCount)
})
})On the client side here is a screenshot of an example where I just duplicated the editor and gave different URLs (http://localhost:6002/doc1 and http://localhost:6002/project1/doc1) to each copy:
Here is the source code: https://gist.github.com/Julien1138/b480927caf65f65c09ed1629591a9505
Creating multiple documents using rooms
So far, we have a functional multi-document collaborative text editor based on tiptap. And for some of you it might be enough to just use namespaces. Or perhaps you don’t want to use namespaces and you just want to use rooms. The following lines detail the implementation of rooms in complement of namespaces, but the same principle can be applied without using namespaces.
This part is a little more complex, because unlike namespaces, where the namespace is defined by the URL on connection, with rooms, nothing tells the server which room the client wants to join. So we have to implement a new event receiver on the server side to allow the client to join a specific room.
So here we go, let’s call it joinRoom:
socket.on('joinRoom', async (room) => {
socket.join(room);
[...]
}The first thing to do in the callback function, is to join the room using the socket.join() function to which we pass the room name.
And for all our other event receivers (update and disconnect) to automatically be assigned to the right room, let’s move their code inside the joinRoom event receiver callback.
Like for namespace, we add a new parameter to file read and write functions to tweak the database file names according to the room they’re assigned to.
Here is what the server code looks like now:
import fs from 'fs'
import { Step } from 'prosemirror-transform'
import schema from './schema.js'
import { join } from 'path'
import { disconnect } from 'process'
// setup socket server
const app = require('express')()
const http = require('http').Server(app)
const io = require('socket.io')(http)
http.listen(6002)
// options
const simulateSlowServerDelay = 0 // milliseconds
const dbPath = './db'
const docTrailer = '-db.json'
const lockedTrailer = '-db_locked.json'
const stepsTrailer = '-db_steps.json'
const maxStoredSteps = 1000
const defaultData = {
"version": 0,
"doc": { "type": "doc", "content": [{ "type": "paragraph", "content":[{ "type": "text", "text": "Let's start collaborating. Yeah!" }] }] }
}
const sleep = (ms) => (new Promise(resolve => setTimeout(resolve, ms)));
function initProjectDir(namespaceDir) {
if (!fs.existsSync(dbPath + namespaceDir)){
fs.mkdirSync(dbPath + namespaceDir, { recursive: true })
}
}
function storeDoc(data, namespaceDir, roomName) {
fs.writeFileSync(dbPath + namespaceDir + '/' + roomName + docTrailer, JSON.stringify(data, null, 2))
}
function storeSteps({steps, version}, namespaceDir, roomName) {
let limitedOldData = []
try {
const oldData = JSON.parse(fs.readFileSync(dbPath + namespaceDir + '/' + roomName + stepsTrailer, 'utf8'))
limitedOldData = oldData.slice(Math.max(oldData.length - maxStoredSteps))
} catch(e) {
}
const newData = [
...limitedOldData,
...steps.map((step, index) => {
return {
step: JSON.parse(JSON.stringify(step)),
version: version + index + 1,
clientID: step.clientID,
}
})
]
fs.writeFileSync(dbPath + namespaceDir + '/' + roomName + stepsTrailer, JSON.stringify(newData))
}
function storeLocked(locked, namespaceDir, roomName) {
fs.writeFileSync(dbPath + namespaceDir + '/' + roomName + lockedTrailer, locked.toString())
}
function getDoc(namespaceDir, roomName) {
try {
return JSON.parse(fs.readFileSync(dbPath + namespaceDir + '/' + roomName + docTrailer, 'utf8'))
} catch(e) {
return defaultData
}
}
function getLocked(namespaceDir, roomName) {
try {
return JSON.parse(fs.readFileSync(dbPath + namespaceDir + '/' + roomName + lockedTrailer, 'utf8'))
} catch(e) {
return false
}
}
function getSteps(version, namespaceDir, roomName) {
try {
const steps = JSON.parse(fs.readFileSync(dbPath + namespaceDir + '/' + roomName + stepsTrailer, 'utf8'))
return steps.filter(step => step.version > version)
} catch(e) {
return []
}
}
const namespaces = io.of(/^\/[a-zA-Z0-9_\/-]+$/)
namespaces.on('connection', socket => {
const namespace = socket.nsp;
const namespaceDir = namespace.name
initProjectDir(namespaceDir)
socket.on('joinRoom', async (room) => {
socket.join(room);
socket.on('update', async ({ version, clientID, steps }) => {
// we need to check if there is another update processed
// so we store a "locked" state
const locked = getLocked(namespaceDir, room)
if (locked) {
// we will do nothing and wait for another client update
return
}
storeLocked(true, namespaceDir, room)
const storedData = getDoc(namespaceDir, room)
await sleep(simulateSlowServerDelay)
// version mismatch: the stored version is newer
// so we send all steps of this version back to the user
if (storedData.version !== version) {
namespace.in(room).emit('update', {
version,
steps: getSteps(version, namespaceDir, room),
})
storeLocked(false, namespaceDir, room)
return
}
let doc = schema.nodeFromJSON(storedData.doc)
await sleep(simulateSlowServerDelay)
let newSteps = steps.map(step => {
const newStep = Step.fromJSON(schema, step)
newStep.clientID = clientID
// apply step to document
let result = newStep.apply(doc)
doc = result.doc
return newStep
})
await sleep(simulateSlowServerDelay)
// calculating a new version number is easy
const newVersion = version + newSteps.length
// store data
storeSteps({ version, steps: newSteps }, namespaceDir, room)
storeDoc({ version: newVersion, doc }, namespaceDir, room)
await sleep(simulateSlowServerDelay)
// send update to everyone (me and others)
namespace.in(room).emit('update', {
version: newVersion,
steps: getSteps(version, namespaceDir, room),
})
storeLocked(false, namespaceDir, room)
})
// send latest document
namespace.in(room).emit('init', getDoc(namespaceDir, room))
// send client count
namespace.in(room).emit('getCount', namespace.adapter.rooms[room].length)
socket.on('disconnect', () => {
if (namespace.adapter.rooms[room]) {
namespace.in(room).emit('getCount', namespace.adapter.rooms[room].length)
}
})
})
})Note that the method to retrieve the number of users connected had to change to be detailed per room.
On the client side, we just have to emit the joinRoom event after the socket creation and voilà:
The source code can be found here : https://gist.github.com/Julien1138/fd6b80dcc2d9cbc0172763167adceaa6
Going further
This article brings simple modifications to the example given by tiptap. It can obviously be improved. The database management functions can be moved in a separate class for example. Other functionalities like displaying the cursors and text selections of other connected users can be added. I’ll let that to you.
I hope you liked this adventure and that I may have helped some of you understanding how a multi-document collaborative tiptap text editor could work.
Thanks for reading me. Have a nice day!
