Perfect — let’s build this the right way.
Since you chose Node.js, I’ll walk you through a clean, powerful structure that will let you:
• Search GitHub for PWAs
• Filter results
• Fork each repo
• Add topics (categories)
• Log everything
• Expand later into a full PWA‑crawler bot
I’ll give you the architecture + starter code, and then you can tell me how far you want to take it.
—
🚀 Overall Architecture (Node.js)
Here’s the structure that works best:
pwa-forker/
├── index.js
├── search.js
├── fork.js
├── categorize.js
├── utils/
│ └── github.js
├── logs/
└── .env
Why this structure?
• Easy to expand
• Each part does one job
• Clean separation of concerns
• Perfect for turning into a GitHub Action later
—
🔑 Step 1 — Install dependencies
You’ll use GitHub’s official API client:
npm init -y
npm install @octokit/rest dotenv
—
🔐 Step 2 — Create `.env`
GITHUB_TOKEN=ghp_your_personal_access_token_here
GITHUB_USERNAME=yourgithubusername
You’ll need a token with:
• repo
• public_repo
permissions.
—
🧠 Step 3 — Create `utils/github.js`
This initializes the GitHub API client:
import { Octokit } from “@octokit/rest”;
import dotenv from “dotenv”;
dotenv.config();
export const octokit = new Octokit({
auth: process.env.GITHUB_TOKEN
});
export const username = process.env.GITHUB_USERNAME;
—
🔍 Step 4 — Create `search.js` (search for PWAs)
This searches GitHub for repos containing PWA files:
import { octokit } from “./utils/github.js”;
export async function searchPWAs(query) {
const results = await octokit.rest.search.repos({
q: `${query} in:path filename:manifest.json`,
per_page: 50
});
return results.data.items;
}
Example queries you can pass:
• “pwa”
• “game pwa”
• “offline pwa”
• “react pwa”
• “vue pwa”
—
🍴 Step 5 — Create `fork.js` (fork repos)
import { octokit, username } from “./utils/github.js”;
export async function forkRepo(fullName) {
const [owner, repo] = fullName.split(“/”);
const res = await octokit.rest.repos.createFork({
owner,
repo
});
console.log(`Forked: ${fullName} → ${username}/${repo}`);
return res.data;
}
—
🏷️ Step 6 — Create `categorize.js` (add topics)
import { octokit, username } from “./utils/github.js”;
export async function addTopics(repo, topics) {
await octokit.rest.repos.replaceAllTopics({
owner: username,
repo,
names: topics
});
console.log(`Added topics to ${repo}: ${topics.join(“, “)}`);
}
—
🧩 Step 7 — Create `index.js` (main script)
This ties everything together:
import { searchPWAs } from “./search.js”;
import { forkRepo } from “./fork.js”;
import { addTopics } from “./categorize.js”;
async function run() {
const categories = [
{ query: “pwa game”, topics: [“pwa”, “game”] },
{ query: “pwa productivity”, topics: [“pwa”, “productivity”] },
{ query: “pwa tools”, topics: [“pwa”, “tools”] }
];
for (const cat of categories) {
console.log(`Searching: ${cat.query}`);
const repos = await searchPWAs(cat.query);
for (const repo of repos) {
const fork = await forkRepo(repo.full_name);
await addTopics(fork.name, cat.topics);
}
}
}
run();
—
🎉 What this script does right now
• Searches GitHub for PWAs by category
• Forks each repo into your account
• Adds category topics
• Logs everything
This is already a PWA‑harvesting bot.
—
🔥 What we can add next
If you want to push this further, we can add:
Advanced features
• Pagination (search beyond 50 results)
• Duplicate detection
• Skip already‑forked repos
• Auto‑sync forks weekly
• Lighthouse scoring for each PWA
• A dashboard of all your PWAs
• A GitHub Action that runs daily
Extreme features
• Crawl PWA directories (Appscope, PWA Rocks)
• Detect real PWAs by scanning repo contents
• Auto‑generate README summaries
• Auto‑tag by framework (React, Vue, Svelte)
• Auto‑tag by capabilities (offline, installable, push)
—
Before we expand it…
Do you want this script to:
A) Run locally only
B) Run automatically on a schedule (GitHub Action)
C) Become a full PWA‑crawler bot with dashboards
Tell me which direction you want to take next and I’ll build it out.
Solar 