const express = require('express') const multiparty = require('multiparty') const path = require('path') const fsa = require('fs') const fs = fsa.promises const sqlite = require('sqlite') const sqlite3 = require('sqlite3') const crypto = require('crypto') const URL = require('url') const router = express.Router() const cfgLoader = require(path.join('..', '..', 'config-loader'))(path.join(__dirname, 'config.json'), { database: 'index.db', root: './files', expiry: 2246400, gateway: 'https://distributed.icynet.eu', shortener: { url: 'https://go.lunasqu.ee', bytes: 2, }, tokens: {} }) function asyncForm (req, form) { return new Promise(function (resolve, reject) { form.parse(req, function(err, fields, files) { if (err) return reject(err) resolve({ fields, files }) }) }) } async function clearDatabase (config, db) { // Remove expired files let files = await db.all('SELECT * FROM File WHERE upload < ?', new Date() - (config.expiry * 1000)) if (files.length > 0) { for (let i in files) { let f = files[i] try { await fs.unlink(path.join(config.root, f.path)) } catch (e) {} await db.run('DELETE FROM File WHERE path = ?', f.path) } } // IPFS hashes let hashes = await db.all('SELECT * FROM Translation WHERE timeat < ?', new Date() - (config.expiry * 1000)) if (hashes.length > 0) { for (let i in hashes) { await db.run('DELETE FROM Translation WHERE file_hash = ?', hashes[i].file_hash) } } // Shortened URLs let shorts = await db.all('SELECT hash FROM Short WHERE timeat < ?', new Date() - (config.expiry * 1000)) if (shorts.length > 0) { for (let i in shorts) { await db.run('DELETE FROM Short WHERE hash = ?', shorts[i].hash) } } console.log('Database was cleared of %d files, %d IPFS hashes and %d shortened URLs.', files.length, hashes.length, shorts.length) } function handleAuthHeader (req, res) { let header = req.header('authorization') if (!header) return null let token = header.split(/\s+/).pop() || '' let auth = Buffer.from(token, 'base64').toString() let parts = auth.split(/:/) return (parts[1] && parts[1] !== '') ? parts[1] : null } async function init () { // Load configuration let config = await cfgLoader let root = path.resolve(config.root) // Check for root directory await fs.access(root, fsa.constants.F_OK) // Initialize database const dbPromise = sqlite.open({ filename: path.join(__dirname, config.database), driver: sqlite3.cached.Database }) let db = await dbPromise await db.migrate({ migrationsPath: path.join(__dirname, 'migrations') }) await clearDatabase(config, db) // Upload file form router.get('/publish', (req, res, next) => { let token = handleAuthHeader(req, res) if (!token || !config.tokens[token]) { return res.status(401).set('WWW-Authenticate', 'Basic realm="Token Auth", charset="UTF-8"').end() } next() }, async (req, res) => { res.sendFile(path.join(__dirname, 'up.html')) }) // Upload a file or publish a hash router.post('/publish', async (req, res, next) => { let ip = req.ip let token = req.header('token') || req.body.token if (req.header('authorization')) token = handleAuthHeader(req, res) if (!token || !config.tokens[token]) return res.status(402).send('Forbidden') let baseurl = config.tokens[token] // Handle IPFS hash let hash = req.query.hash || req.body.hash if (hash && hash !== '') { let filename = req.query.filename || req.body.filename if (!filename) filename = crypto.randomBytes(8).toString('hex') let db = await dbPromise await db.run('INSERT INTO Translation (translation,file_hash,timeat) VALUES (?,?,?)', filename, hash, new Date()) return res.send(baseurl + filename) } // Handle multipart data let form = new multiparty.Form() let { fields, files } = await asyncForm(req, form) // Detect all files let allFiles = [] for (let i in files) { for (let j in files[i]) { allFiles.push(files[i][j]) } } if (!allFiles.length) return res.status(400).send('Invalid request') console.log('[%s] from %s request to upload %d file(s)', new Date(), ip, allFiles.length) // Handle all files provided let db = await dbPromise let uploadedFiles = [] for (let i in allFiles) { let file = allFiles[i] let fname = file.originalFilename let target = path.join(root, fname) // Handle already exists case (overwrite) try { await fs.access(target, fsa.constants.F_OK) await fs.unlink(target) await fs.copyFile(file.path, target) await fs.unlink(file.path) await db.run('UPDATE File SET ip = ?, upload = ? WHERE path = ?', ip, new Date(), fname) uploadedFiles.push(baseurl + fname) continue } catch (e) { if (e.code !== 'ENOENT') throw e } // Copy to target and unlink temporary file await fs.copyFile(file.path, target) await fs.unlink(file.path) await db.run('INSERT INTO File (path,ip,upload) VALUES (?,?,?)', fname, ip, new Date()) uploadedFiles.push(baseurl + fname) } if (uploadedFiles.length === 0) return res.status(400).send('No files were uploaded') let tagify = fields && fields['tagify'] if (tagify != null) { let a = uploadedFiles.length > 1 ? ' target="_blank"' : '' for (let i in uploadedFiles) { uploadedFiles[i] = '' + uploadedFiles[i] + '' } } res.send(uploadedFiles.join(tagify ? '
' : '\n')) }) // Shorten URL form router.get('/shorten', async (req, res) => { res.sendFile(path.join(__dirname, 'go.html')) }) // Shorten URLs router.post('/shorten', async (req, res) => { let ip = req.ip let url = req.body.url // Simple URL validator try { let a = new URL.URL(url) if (a.protocol.indexOf('http') !== 0 && a.protocol.indexOf('ftp') !== -1) { throw new Error('Unsupported protocol') } } catch (e) { throw new Error('Invalid URL!') } let db = await dbPromise let use let existing = await db.get('SELECT hash FROM Short WHERE url = ?', url) if (existing) { use = existing.hash } else { // Get a hash that isnt in use for (let i = 0; i < 8; i++) { let add = Math.floor(i / 2) let hash = crypto.randomBytes((config.shortener.bytes || 2) + add).toString('hex') let exists = await db.get('SELECT timeat FROM Short WHERE hash = ?', hash) if (!exists) { use = hash break } } if (!use) throw new Error('Server could not find a proper hash for some reason') await db.run('INSERT INTO Short (url,hash,timeat,ip) VALUES (?,?,?,?)', url, use, Date.now(), ip) } let ua = req.get('User-Agent') let reqRaw = false if (!reqRaw && ua && (ua.match(/curl\//i) != null || ua.match(/wget\//i) != null)) reqRaw = true let resp = config.shortener.url + '/' + use if (!reqRaw) resp = '' + resp + '' res.send(resp) }) router.get('/shorten/:hash', async (req, res) => { let hash = req.params.hash let db = await dbPromise let get = await db.get('SELECT url FROM Short WHERE hash = ?', hash) if (!get) throw new Error('No such hash exists in the database.') res.redirect(get.url) }) // Serve a file or a hash // Files should be served from an external web server (such as nginx) whenever possible. router.get('/:hash', async (req, res, next) => { if (!req.params.hash) return res.status(400).send('Invalid request') let db = await dbPromise let file = await db.get('SELECT * FROM File WHERE path = ?', req.params.hash) let translation = await db.get('SELECT * FROM Translation WHERE translation = ?', req.params.hash) if (!file && !translation) return res.status(404).end() res.header('Cache-Control', 'max-age=' + 7 * 24 * 60 * 60 * 1000) if (translation) { return res.redirect(config.gateway + '/ipfs/' + translation.file_hash) } res.sendFile(path.join(root, file.path)) }) return router } module.exports = init