Document 01 / 03
Project Status Report
1. Executive Summary#
Tunnel is a full-stack live streaming platform (React 19 + Vite frontend, Express 5 + MongoDB backend, Node-Media-Server for RTMP/HLS, Socket.IO for realtime, Clerk for auth, Razorpay for payments, Gemini for AI features).
Headline finding: the platform is well-scaffolded but most critical end-to-end flows are broken. The single most damaging defect is that stream creation saves a Clerk string ID (user_2...) into a Mongoose ObjectId field (stream.creator), which throws a CastError, so no stream can ever be created. Every feature that depends on streams (Go Live, Explore, Dashboard, Watch, chat, analytics, recording) is downstream of this and therefore non-functional today.
Updated completion estimate: ~30–35% of the features claimed in README.md/PRD_AND_ROADMAP.md actually work end-to-end. The previous PRD estimate of 58% (Backend 65%) was too optimistic.
2. Verified Completion Matrix (what actually works vs. what is broken)#
| Area | Est. | Verified status |
|---|---|---|
| Clerk auth (sign-in, token plumbing) | 70% | Works: main.jsx → ClerkProvider + setClerkTokenProvider → apiClient attaches Bearer token; backend clerkMiddleware parses it. But custom require-Auth.js middleware does NOT verify the JWT signature (see §4 and AUTHENTICATION_GUIDE.md). |
| Streaming / RTMP / HLS | 25% | NMS starts on ports 1935/8080; socket handlers exist. But stream creation crashes (CastError), goLive is not wired to any route, the frontend never emits stream-started, so isLive never becomes true, /streams/live is always empty, and the returned HLS URL is wrong (/hls/{key}.m3u8 vs NMS's real /live/{key}/index.m3u8). |
| Realtime chat | 45% | Socket send-message flow exists with moderation/rate-limit/slow-mode, but Message.create passes Clerk string sender + Mongo category into ObjectId fields → CastError → messages never persist. REST /chat/messages/:streamId has the same category type problem. |
| Payments (Razorpay) | 55% | Backend payment.controller.js is solid (create-order, verify, refund, webhook, history, stats). Frontend wiring is broken: payment.jsx calls paymentAPI.createPayment(...) / deletePayment(...) which do not exist in apiClient.js → TypeError. |
| Analytics | 35% | Backend report generator (generateAnalyticsReport) is robust and handles Clerk→MongoId conversion, but no analytics records are ever created (nothing calls POST /analytics/create, and stream creation is broken anyway). Dashboard passes no chart data → charts empty/"No data available". profile.controller getAnalytics reads nonexistent stream fields (startTime/endTime/totalMessages) and silently falls back to random mock data. |
| Profile / Settings | 45% | GET /auth/profile works (auto-creates user). profile.controller endpoints at /profile/:userId crash for Clerk IDs (User.findById(clerkId) → CastError). GET /auth/settings does not exist (only PUT) → settings page can't load saved state. |
| Follow / Social | 10% | Follow controller uses User.findById with Clerk strings → CastError; increments nonexistent followingcount/followerscount; checks nonexistent blockedUsers. Social.controller.js handlers are empty stubs. |
| AI (Gemini) | 40% | gemini.controller.js works when GEMINI_API_KEY is set (streaming system prompt, chat, recommendations). But the chat widget uses useChatbot.js which reads a legacy localStorage token that is never set in the Clerk flow → chatbot requests fail auth. |
| VOD / Recording | 20% | ffmpeg-recorder.js is well built but is only triggered by the stream-started socket event, which is never emitted. vodAPI.getAll() hits /streams/vod/all — a route that does not exist → 404. No express.static serves the recorded files. |
| DevOps / Deploy | 15% | No Dockerfile, no CI, no express.static, hardcoded localhost:5000/8080/1935 throughout backend responses and frontend socket/player code. Frontend only deploys to Vercel (SPA rewrite). |
| Testing | 10% | Tests exist (backend/src/__tests__/api-timeout.test.js, razorpay-config.test.js, validation.middleware.test.js; Frontend/__tests__/auth-flow.test.jsx) but backend npm test is echo "no test specified" && exit 1 and the frontend has no test script — tests can't run. |
3. Architecture (verified, matches code)#
Browser (React 19 SPA, Vite 7, Tailwind 4)
│ REST → http://localhost:5000/api/v1/* (axios + Clerk Bearer token)
│ WS → socket.io http://localhost:5000
│ HLS → http://localhost:8080/live/{streamKey}/index.m3u8
▼
Express 5 app (app.js, server.js)
├─ CORS (origins from CORS_ORIGIN, default localhost:5173)
├─ requestLogger + requestTimeoutGuard(30s) + res.setTimeout(8s)
├─ clerkMiddleware() (@clerk/express)
├─ checkDBConnection
├─ Global rate limiter: 100 req / 15 min per IP on every /api/v1 mount
├─ Routes: /auth /streams /profile /payment /analytics /follower /chat /gemini /social
├─ Diagnostic endpoints: /api/health, /api/auth/debug/setup, /api/v1/profile/debug,
│ /api/v1/analytics/debug, /api/v1/analytics/debug/auth
└─ Socket.IO (cors "*") + Node-Media-Server (RTMP 1935, HLS 8080, mediaroot ./media)
└─ FFmpeg recording util (ffmpeg-recorder.js) — triggered only by socket "stream-started"
MongoDB (dbconnect.js: 3 retries, 5s delay, degraded mode if down)
Models (all exist and are indexed)#
User— clerkId (unique), email, username, role (viewer/creator/moderator/admin), notifications, settings, security, apiKeys.Stream— creator (ObjectId ref), streamKey (unique), isLive, viewers, recording/VOD fields, startedAt/endedAt. NostartTime/endTime/totalMessagesfields (several controllers read these — see bugs).Analytics,Payment,ChatMessage(categoryis ObjectId ref to Stream),ChatModeration,FollowerRelationShip(follower/followingObjectId refs).
4. Critical Known Problems (with evidence)#
P0 — Blocks the entire product#
-
Stream creation is broken (CastError).
stream.controller.js:24storescreator: userIdwhereuserId = req.auth.userIdis a Clerk string (user_2...), but the schema field isObjectId+required. Mongoose throwsCastError: Cast to ObjectId failed for value "user_2..."→POST /streams/createreturns 500. No stream can ever be created, so nothing downstream works. -
goLiveis dead code +isLivenever set.goLive(stream.controller.js:49) is not mounted on any route.golive.jsxnever emits the socketstream-startedevent thatsockets.js:225listens for. NMSprePublish/donePublishonlyconsole.log. Result:Stream.isLiveis nevertrue,/streams/livereturns[], Explore/Dashboard/Watch show nothing. -
Wrong HLS URL served to viewers.
createStream(line 33) andgetStreamById(line 131) returnhttp://localhost:8080/hls/{streamKey}.m3u8, but Node-Media-Server actually serves/live/{streamKey}/index.m3u8. The Watch page (profile/stream.jsx:118) passesstream.hlsUrlintoStreamPlayerasoverrideHlsUrl, so even with a live stream, playback fails. (golive.jsxuses the correct URL, which is why the creator's own preview works.) -
Auth security hole —
require-Auth.jsdoes not verify signatures. It onlyjwtDecode()s the token and checks issuer contains "clerk" + expiry. A forged/unsigned JWT withsubset andisscontaining "clerk" passes. Any route using the customrequireAuth(auth, profile, payment, analytics, chat, follower) is spoofable. The@clerk/expressrequireAuth(used bystream.route.js,Social.controller.js) does real verification — two inconsistent auth stacks. Fix: replace custom middleware with@clerk/expressrequireAuth(). -
Follow system entirely broken.
follow.controller.jsusesUser.findById(followId)/findById(followingId)with Clerk strings → CastError;$incon nonexistentfollowingcount/followerscount; references nonexistentblockedUsers;Folllower.creategets Clerk strings into ObjectId fields. FrontendFollow.checkFollowsends onlytargetUserIdbutcheckFollowStatusrequires bothuserIdandtargetUserId.
P1 — Major#
-
Payment frontend calls missing API methods.
pages/payment.jsx:48,63callpaymentAPI.createPaymentandpaymentAPI.deletePayment, which are not inapiClient.js→TypeError: paymentAPI.createPayment is not a function. AlsoPaymentForm.jsximportsuseAuthfrom@clerk/clerk-react(v5) while the rest of the app uses@clerk/react(v6) — both packages are installed; inconsistent and fragile. -
Settings page can't load.
settingsAPI.getSettings()→GET /auth/settings→ route does not exist (user.route.jsonly definesPUT /settings). Page falls back to defaults; toggles update via PUT but never reflect saved state. Two schema shapes conflict:user.controller.updateSettingswritessettings.themeetc.,profile.controller.updateUserSettingswritesnotifications.twoFactorEnabled. -
VOD page 404.
vodAPI.getAll()→GET /streams/vod/all→ no such route (actual:/creator/:creatorId/recordings). Also no static server for recorded media. -
Chat messages don't persist.
sockets.jsMessage.create({ sender: <clerk string>, category: <streamId string> })→ both are ObjectId schema fields → CastError → messages silently dropped. Same for RESTgetChatMessages. -
Dashboard/analytics are empty/decorative.
DashboardPage.jsxpasses nochartData/trafficSourcestoDashboardContent, so Recharts render empty states.profile.controller.getAnalyticsreads nonexistent stream fields and returns random mock data when no streams exist (which is always).
P2 — Quality / production-readiness#
-
Hardcoded localhost everywhere —
hlsUrl/rtmpUrlin controllers,io("http://localhost:5000")ingolive.jsx/stream.jsx/DashboardPage.jsx/layouts,StreamPlayerdefaulthttp://localhost:8080/live/.... No env-driven base URLs → cannot deploy to any host as-is.Frontend/.envusesVITE_API_URL=http://localhost:5000/api/v1. -
NMS stream keys not validated.
prePublish/donePublishdon't check the stream key against the DB, so anyone can publish to any key, and there's no mapping between the RTMP publish event andStream.isLive. Noon_publishwebhook → auth/user state is never reconciled server-side. -
Social endpoints are empty stubs (
controller/Social.controller.js— all handlers empty try/catch), and routes lack leading slashes (router.post("user/:id/like")→/api/v1/social/user/:id/like). -
WebSocket is unauthenticated.
iohascors: { origin: "*" }andsend-messagetrusts a client-supplieduserId— trivial to spoof as another user or flood chat (only a per-socket in-memory rate limit exists). -
express.staticmissing formedia/,recordings/,hls/,vod/— NMS media root and recording outputs are not served by the Express app. -
Tests can't run. Backend
npm testis a placeholder; frontend hasjestinstalled but notestscript. -
Env hygiene.
backend/src/.envexists untracked (contains real secrets);Frontend/.envis tracked in git (contains Clerk publishable key — public by design, butVITE_API_URLis localhost). No.env.examplefor frontend; the README documents variable names that don't match the code (VITE_API_BASE_URLvs actualVITE_API_URL,VITE_SOCKET_URLunused).
5. Recommended Priority Fix Order (toward a working MVP)#
Phase 1 — unblock core streaming (1–2 days)
- Fix
createStream: resolveclerkId → ObjectId(User.findOne({ clerkId })), store that increator. (This one fix unblocks Go Live, Explore, Dashboard, Watch, chat, analytics.) - Wire
goLiveroute or emitstream-startedfromgolive.jsxafter create + have NMSprePublishfire it (webhook-style), soisLive: truegets set. - Fix HLS URLs to
/live/{streamKey}/index.m3u8(and make them env-driven:HLS_BASE_URL).
Phase 2 — fix auth + payments + profiles (1–2 days)
4. Replace custom require-Auth.js with @clerk/express requireAuth() on all routes; delete the custom one. Add a shared helper to convert Clerk ID → Mongo _id (used by stream/follow/profile/chat).
5. Fix follow controller (ObjectId resolution, real counters on the schema), fix checkFollowStatus query contract.
6. Add createPayment/deletePayment to apiClient.js (or fix payment.jsx to use existing methods). Unify Clerk package usage (@clerk/react).
7. Add GET /auth/settings and align the settings schema with the frontend toggles.
Phase 3 — chat, analytics, VOD, deployment (2–3 days)
8. Fix Message save to convert sender/category to ObjectIds (or make chat schema store strings).
9. Create Analytics records from the stream lifecycle (start/end + engagement events) so dashboards/reports have real data; remove mock-data fallback.
10. Fix VOD route + vodAPI, add express.static for media dirs, wire FFmpeg recording to prePublish/donePublish.
11. Drive all base URLs from env (frontend VITE_*, backend HLS_BASE_URL/RTMP_BASE_URL); add Dockerfile + docker-compose; un-ignore .env correctly and add .env.example.
Phase 4 — hardening
12. Websocket auth (Clerk session token in socket auth, verify on server), NMS stream-key validation, real rate limiting per user on sockets, remove/deactivate empty social stubs, enable npm test for both packages and add CI.
6. What Works Today (verified)#
- Clerk sign-in/up and session;
GET /auth/profileauto-creates the Mongo user (race-condition safe). - Backend boots with graceful degraded mode when MongoDB is down; global 8s/30s timeout guards; rich request logging.
- NMS RTMP + HLS servers start on 1935/8080 (if ports free); a manual OBS publish to the correct key is actually viewable in the creator preview (
golive.jsxuses the right URL). - Socket viewer counting, metrics broadcast, and moderation logic are implemented server-side (they just don't persist messages).
- Razorpay backend flow (create-order → verify → refund → webhook) is complete and correct.
- Gemini chat/recommendations/title/description endpoints work when the API key is set.
generateAnalyticsReportis production-quality (timeout failsafe, Clerk→MongoId conversion, rich report).- Frontend UI: polished layouts, sidebar/navbar, AboutUs, auth screens, explore grid, stream page skeleton, payment history/stats cards.
This report was generated from a full source audit. Fix the P0 items first — they cascade into everything else. See AUTHENTICATION_GUIDE.md and STREAMING_IMPLEMENTATION.md for detailed deep-dives.