Document 01 / 03

Project Status Report

Version 2.0August 14, 20269 min readSource on GitHub

1. Executive Summary#

Tunnel is a full-stack live streaming platform (React 19 + Vite frontend, Express 5 + MongoDB backend, Node-Media-Server for RTMP/HLS, Socket.IO for realtime, Clerk for auth, Razorpay for payments, Gemini for AI features).

Headline finding: the platform is well-scaffolded but most critical end-to-end flows are broken. The single most damaging defect is that stream creation saves a Clerk string ID (user_2...) into a Mongoose ObjectId field (stream.creator), which throws a CastError, so no stream can ever be created. Every feature that depends on streams (Go Live, Explore, Dashboard, Watch, chat, analytics, recording) is downstream of this and therefore non-functional today.

Updated completion estimate: ~30–35% of the features claimed in README.md/PRD_AND_ROADMAP.md actually work end-to-end. The previous PRD estimate of 58% (Backend 65%) was too optimistic.


2. Verified Completion Matrix (what actually works vs. what is broken)#

AreaEst.Verified status
Clerk auth (sign-in, token plumbing)70%Works: main.jsx → ClerkProvider + setClerkTokenProvider → apiClient attaches Bearer token; backend clerkMiddleware parses it. But custom require-Auth.js middleware does NOT verify the JWT signature (see §4 and AUTHENTICATION_GUIDE.md).
Streaming / RTMP / HLS25%NMS starts on ports 1935/8080; socket handlers exist. But stream creation crashes (CastError), goLive is not wired to any route, the frontend never emits stream-started, so isLive never becomes true, /streams/live is always empty, and the returned HLS URL is wrong (/hls/{key}.m3u8 vs NMS's real /live/{key}/index.m3u8).
Realtime chat45%Socket send-message flow exists with moderation/rate-limit/slow-mode, but Message.create passes Clerk string sender + Mongo category into ObjectId fields → CastError → messages never persist. REST /chat/messages/:streamId has the same category type problem.
Payments (Razorpay)55%Backend payment.controller.js is solid (create-order, verify, refund, webhook, history, stats). Frontend wiring is broken: payment.jsx calls paymentAPI.createPayment(...) / deletePayment(...) which do not exist in apiClient.js → TypeError.
Analytics35%Backend report generator (generateAnalyticsReport) is robust and handles Clerk→MongoId conversion, but no analytics records are ever created (nothing calls POST /analytics/create, and stream creation is broken anyway). Dashboard passes no chart data → charts empty/"No data available". profile.controller getAnalytics reads nonexistent stream fields (startTime/endTime/totalMessages) and silently falls back to random mock data.
Profile / Settings45%GET /auth/profile works (auto-creates user). profile.controller endpoints at /profile/:userId crash for Clerk IDs (User.findById(clerkId) → CastError). GET /auth/settings does not exist (only PUT) → settings page can't load saved state.
Follow / Social10%Follow controller uses User.findById with Clerk strings → CastError; increments nonexistent followingcount/followerscount; checks nonexistent blockedUsers. Social.controller.js handlers are empty stubs.
AI (Gemini)40%gemini.controller.js works when GEMINI_API_KEY is set (streaming system prompt, chat, recommendations). But the chat widget uses useChatbot.js which reads a legacy localStorage token that is never set in the Clerk flow → chatbot requests fail auth.
VOD / Recording20%ffmpeg-recorder.js is well built but is only triggered by the stream-started socket event, which is never emitted. vodAPI.getAll() hits /streams/vod/all — a route that does not exist → 404. No express.static serves the recorded files.
DevOps / Deploy15%No Dockerfile, no CI, no express.static, hardcoded localhost:5000/8080/1935 throughout backend responses and frontend socket/player code. Frontend only deploys to Vercel (SPA rewrite).
Testing10%Tests exist (backend/src/__tests__/api-timeout.test.js, razorpay-config.test.js, validation.middleware.test.js; Frontend/__tests__/auth-flow.test.jsx) but backend npm test is echo "no test specified" && exit 1 and the frontend has no test script — tests can't run.

3. Architecture (verified, matches code)#

Browser (React 19 SPA, Vite 7, Tailwind 4)
   │  REST  →  http://localhost:5000/api/v1/*   (axios + Clerk Bearer token)
   │  WS    →  socket.io http://localhost:5000
   │  HLS   →  http://localhost:8080/live/{streamKey}/index.m3u8
   ▼
Express 5 app (app.js, server.js)
   ├─ CORS (origins from CORS_ORIGIN, default localhost:5173)
   ├─ requestLogger + requestTimeoutGuard(30s) + res.setTimeout(8s)
   ├─ clerkMiddleware() (@clerk/express)
   ├─ checkDBConnection
   ├─ Global rate limiter: 100 req / 15 min per IP on every /api/v1 mount
   ├─ Routes: /auth /streams /profile /payment /analytics /follower /chat /gemini /social
   ├─ Diagnostic endpoints: /api/health, /api/auth/debug/setup, /api/v1/profile/debug,
   │     /api/v1/analytics/debug, /api/v1/analytics/debug/auth
   └─ Socket.IO (cors "*") + Node-Media-Server (RTMP 1935, HLS 8080, mediaroot ./media)
        └─ FFmpeg recording util (ffmpeg-recorder.js) — triggered only by socket "stream-started"
MongoDB (dbconnect.js: 3 retries, 5s delay, degraded mode if down)

Models (all exist and are indexed)#

  • User — clerkId (unique), email, username, role (viewer/creator/moderator/admin), notifications, settings, security, apiKeys.
  • Stream — creator (ObjectId ref), streamKey (unique), isLive, viewers, recording/VOD fields, startedAt/endedAt. No startTime/endTime/totalMessages fields (several controllers read these — see bugs).
  • Analytics, Payment, ChatMessage (category is ObjectId ref to Stream), ChatModeration, FollowerRelationShip (follower/following ObjectId refs).

4. Critical Known Problems (with evidence)#

P0 — Blocks the entire product#

  1. Stream creation is broken (CastError). stream.controller.js:24 stores creator: userId where userId = req.auth.userId is a Clerk string (user_2...), but the schema field is ObjectId + required. Mongoose throws CastError: Cast to ObjectId failed for value "user_2..." → POST /streams/create returns 500. No stream can ever be created, so nothing downstream works.

  2. goLive is dead code + isLive never set. goLive (stream.controller.js:49) is not mounted on any route. golive.jsx never emits the socket stream-started event that sockets.js:225 listens for. NMS prePublish/donePublish only console.log. Result: Stream.isLive is never true, /streams/live returns [], Explore/Dashboard/Watch show nothing.

  3. Wrong HLS URL served to viewers. createStream (line 33) and getStreamById (line 131) return http://localhost:8080/hls/{streamKey}.m3u8, but Node-Media-Server actually serves /live/{streamKey}/index.m3u8. The Watch page (profile/stream.jsx:118) passes stream.hlsUrl into StreamPlayer as overrideHlsUrl, so even with a live stream, playback fails. (golive.jsx uses the correct URL, which is why the creator's own preview works.)

  4. Auth security hole — require-Auth.js does not verify signatures. It only jwtDecode()s the token and checks issuer contains "clerk" + expiry. A forged/unsigned JWT with sub set and iss containing "clerk" passes. Any route using the custom requireAuth (auth, profile, payment, analytics, chat, follower) is spoofable. The @clerk/express requireAuth (used by stream.route.js, Social.controller.js) does real verification — two inconsistent auth stacks. Fix: replace custom middleware with @clerk/express requireAuth().

  5. Follow system entirely broken. follow.controller.js uses User.findById(followId)/findById(followingId) with Clerk strings → CastError; $inc on nonexistent followingcount/followerscount; references nonexistent blockedUsers; Folllower.create gets Clerk strings into ObjectId fields. Frontend Follow.checkFollow sends only targetUserId but checkFollowStatus requires both userId and targetUserId.

P1 — Major#

  1. Payment frontend calls missing API methods. pages/payment.jsx:48,63 call paymentAPI.createPayment and paymentAPI.deletePayment, which are not in apiClient.js → TypeError: paymentAPI.createPayment is not a function. Also PaymentForm.jsx imports useAuth from @clerk/clerk-react (v5) while the rest of the app uses @clerk/react (v6) — both packages are installed; inconsistent and fragile.

  2. Settings page can't load. settingsAPI.getSettings() → GET /auth/settings → route does not exist (user.route.js only defines PUT /settings). Page falls back to defaults; toggles update via PUT but never reflect saved state. Two schema shapes conflict: user.controller.updateSettings writes settings.theme etc., profile.controller.updateUserSettings writes notifications.twoFactorEnabled.

  3. VOD page 404. vodAPI.getAll() → GET /streams/vod/all → no such route (actual: /creator/:creatorId/recordings). Also no static server for recorded media.

  4. Chat messages don't persist. sockets.js Message.create({ sender: <clerk string>, category: <streamId string> }) → both are ObjectId schema fields → CastError → messages silently dropped. Same for REST getChatMessages.

  5. Dashboard/analytics are empty/decorative. DashboardPage.jsx passes no chartData/trafficSources to DashboardContent, so Recharts render empty states. profile.controller.getAnalytics reads nonexistent stream fields and returns random mock data when no streams exist (which is always).

P2 — Quality / production-readiness#

  1. Hardcoded localhost everywhere — hlsUrl/rtmpUrl in controllers, io("http://localhost:5000") in golive.jsx/stream.jsx/DashboardPage.jsx/layouts, StreamPlayer default http://localhost:8080/live/.... No env-driven base URLs → cannot deploy to any host as-is. Frontend/.env uses VITE_API_URL=http://localhost:5000/api/v1.

  2. NMS stream keys not validated. prePublish/donePublish don't check the stream key against the DB, so anyone can publish to any key, and there's no mapping between the RTMP publish event and Stream.isLive. No on_publish webhook → auth/user state is never reconciled server-side.

  3. Social endpoints are empty stubs (controller/Social.controller.js — all handlers empty try/catch), and routes lack leading slashes (router.post("user/:id/like") → /api/v1/social/user/:id/like).

  4. WebSocket is unauthenticated. io has cors: { origin: "*" } and send-message trusts a client-supplied userId — trivial to spoof as another user or flood chat (only a per-socket in-memory rate limit exists).

  5. express.static missing for media/, recordings/, hls/, vod/ — NMS media root and recording outputs are not served by the Express app.

  6. Tests can't run. Backend npm test is a placeholder; frontend has jest installed but no test script.

  7. Env hygiene. backend/src/.env exists untracked (contains real secrets); Frontend/.env is tracked in git (contains Clerk publishable key — public by design, but VITE_API_URL is localhost). No .env.example for frontend; the README documents variable names that don't match the code (VITE_API_BASE_URL vs actual VITE_API_URL, VITE_SOCKET_URL unused).


Phase 1 — unblock core streaming (1–2 days)

  1. Fix createStream: resolve clerkId → ObjectId (User.findOne({ clerkId })), store that in creator. (This one fix unblocks Go Live, Explore, Dashboard, Watch, chat, analytics.)
  2. Wire goLive route or emit stream-started from golive.jsx after create + have NMS prePublish fire it (webhook-style), so isLive: true gets set.
  3. Fix HLS URLs to /live/{streamKey}/index.m3u8 (and make them env-driven: HLS_BASE_URL).

Phase 2 — fix auth + payments + profiles (1–2 days) 4. Replace custom require-Auth.js with @clerk/express requireAuth() on all routes; delete the custom one. Add a shared helper to convert Clerk ID → Mongo _id (used by stream/follow/profile/chat). 5. Fix follow controller (ObjectId resolution, real counters on the schema), fix checkFollowStatus query contract. 6. Add createPayment/deletePayment to apiClient.js (or fix payment.jsx to use existing methods). Unify Clerk package usage (@clerk/react). 7. Add GET /auth/settings and align the settings schema with the frontend toggles.

Phase 3 — chat, analytics, VOD, deployment (2–3 days) 8. Fix Message save to convert sender/category to ObjectIds (or make chat schema store strings). 9. Create Analytics records from the stream lifecycle (start/end + engagement events) so dashboards/reports have real data; remove mock-data fallback. 10. Fix VOD route + vodAPI, add express.static for media dirs, wire FFmpeg recording to prePublish/donePublish. 11. Drive all base URLs from env (frontend VITE_*, backend HLS_BASE_URL/RTMP_BASE_URL); add Dockerfile + docker-compose; un-ignore .env correctly and add .env.example.

Phase 4 — hardening 12. Websocket auth (Clerk session token in socket auth, verify on server), NMS stream-key validation, real rate limiting per user on sockets, remove/deactivate empty social stubs, enable npm test for both packages and add CI.


6. What Works Today (verified)#

  • Clerk sign-in/up and session; GET /auth/profile auto-creates the Mongo user (race-condition safe).
  • Backend boots with graceful degraded mode when MongoDB is down; global 8s/30s timeout guards; rich request logging.
  • NMS RTMP + HLS servers start on 1935/8080 (if ports free); a manual OBS publish to the correct key is actually viewable in the creator preview (golive.jsx uses the right URL).
  • Socket viewer counting, metrics broadcast, and moderation logic are implemented server-side (they just don't persist messages).
  • Razorpay backend flow (create-order → verify → refund → webhook) is complete and correct.
  • Gemini chat/recommendations/title/description endpoints work when the API key is set.
  • generateAnalyticsReport is production-quality (timeout failsafe, Clerk→MongoId conversion, rich report).
  • Frontend UI: polished layouts, sidebar/navbar, AboutUs, auth screens, explore grid, stream page skeleton, payment history/stats cards.

This report was generated from a full source audit. Fix the P0 items first — they cascade into everything else. See AUTHENTICATION_GUIDE.md and STREAMING_IMPLEMENTATION.md for detailed deep-dives.