The API: Claude as a component

Everything in this series ran through claude.ai. The Claude API exposes the same models programmatically — Claude becomes a building block inside your own scripts, apps, and pipelines.

When does the API beat the chat interface?

• Repetition: the same operation on 500 inputs (classify tickets, summarize reviews, extract fields from invoices)

• Integration: Claude inside your product or internal tool

• Scheduling: jobs that run without a human present

• Precision: exact control over the system prompt, model, temperature, and output format

The core call is disarmingly small — send messages, receive a response:

import anthropic

client = anthropic.Anthropic()  # uses ANTHROPIC_API_KEY

response = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1000,
    system="You extract invoice data. Respond with JSON only.",
    messages=[{"role": "user", "content": invoice_text}],
)
print(response.content[0].text)

Wrap that in a loop and you've automated a job. Everything you learned in Part 3 — roles, examples, structured output — is API prompt design; the skills transfer one-to-one. Start at platform.claude.com (new accounts get a small free credit) and the docs at docs.claude.com.

Three API-specific concepts to know exist (learn them when needed): system prompts (your Project instructions, as a parameter), tool use / function calling (Claude decides when to call your functions — how agents are built), and batch processing (bulk jobs at lower cost).

Automation without code

No programming required to automate:

• Connectors + Projects (Part 6) already remove most repetitive context work

• Automation platforms (Zapier, Make, n8n) offer Claude steps: "when a form is submitted → Claude summarizes → result lands in Slack"

• A disciplined prompt library (Part 3) is automation of thought — never solve the same prompt twice

Plans, limits, and economics

As of mid-2026: Free (permanent, real plan — core features included, lower usage caps), Pro (~$20/mo — top models, ~5x usage, Claude Code), Max ($100–200/mo — same features, much higher headroom), plus Team/Enterprise. The API is billed separately, per token.

The professional intuition: chat plans are flat-rate thinking time; the API is metered production. Interactive work belongs in chat; volume work belongs in the API. Details drift — verify at claude.com/pricing.

Stretching limits (now with full context from the series): usage scales with tokens processed, so long conversations are expensive by construction → distill and restart (Part 2); match model size to task difficulty (Part 2); put stable context in Projects rather than re-pasting it (Part 6).

The verification discipline

The habit separating professionals from the burned: calibrated trust. A working checklist —

• Trust freely: brainstorming, drafts, explanations of well-known concepts, code you will execute anyway (the run is the check)

• Verify before use: specific facts, statistics, quotes, citations, URLs, legal/medical/financial claims

• Force verifiability: ground answers in uploaded documents ("quote the section" — Part 4), compute with code, run the reversal ("argue against your answer" — Part 3)

• Never outsource: the final judgment call. Claude informs decisions; it doesn't own them.

Hallucination isn't a reason to avoid these tools — it's a parameter to engineer around. That mindset is professional AI usage.

Staying current

This field moves monthly. Low-effort ways to keep your edge: anthropic.com/news for releases, docs.claude.com when features shift, and one honest hour of experimenting when something new ships. Principles in this series (context, grounding, iteration, verification) age slowly; feature details age fast.

The mastery map — the whole series in one view

• Foundations: tokens, context window, tools vs. training (Part 1)

• Mechanics: context hygiene, edit-don't-pile, model choice (Part 2)

• Prompting: role/task/context/format → examples, structure, reversal, meta-prompting (Part 3)

• Grounding: documents, vision, computed analysis (Part 4)

• Deliverables: Artifacts, iteration, publishing (Part 5)

• Infrastructure: Projects, memory, connectors (Part 6)

• Agency: Claude Code, plan-first, review discipline (Part 7)

• Scale: API, automation, calibrated trust (Part 8)

If you did the exercises, you didn't read about this — you did it. That was the point.

Final exercise

Pick one recurring task from your actual work. Design the full stack for it: which Project, which knowledge files, which prompt template, chat or API, and what verification step. Build it this week. Then teach it to one colleague — teaching is the last stage of mastery.

Thanks for reading the series. Now go build something. 🚀

Mode 1: Chat-assisted coding, done properly

The four core moves — generate, debug, explain, review — you likely know. What separates effective use is how:

Debugging: give the full picture. The highest-value debugging prompt contains three things: the complete error message (or a screenshot — vision works), the relevant code, and what you already tried. That last one prevents Claude from suggesting the fix you've already ruled out.

State your environment, always. "React 18 + TypeScript, no external libraries, must support older browsers" transforms answer quality. Put your permanent stack in Project instructions (Part 6) so you never repeat it.

Review with a rubric. "Review this" gets generic praise. "Review for: security issues, error handling gaps, and anything that will confuse the next developer — cite line numbers" gets a real review. Then run the reversal from Part 3: "now argue why your suggested refactor might be a mistake."

Learn, don't just paste. The prompt "add comments explaining the why of each block, and tell me one concept I should study" turns every answer into a lesson. Beginners: combined with Artifacts (Part 5), you can build and understand real projects from day one.

Verify by execution. Claude's code is good but not guaranteed. The loop is: run it → paste any error back → repeat. Fast, and honest.

Mode 2: Claude Code — the agentic shift

Chat coding has a ceiling: you are the hands, copying snippets file by file. Claude Code removes it. It's a tool from Anthropic that runs in your terminal (with desktop and mobile apps too, and IDE integrations) with direct access to your project. Given a task, it:

1. Explores your codebase — reads files, understands structure

2. Plans the change and shows you the plan

3. Executes — edits multiple files, runs commands and tests

4. Self-corrects — sees failing tests and fixes its own work

5. Reports — you review the diff, like a pull request from a colleague

The difference in kind: "add password-reset flow to this app" is one instruction, not forty copy-pastes.

Getting started: requires a paid Claude plan or API access. Install via npm (npm install -g @anthropic-ai/claude-code), run claude inside your repo. Current setup details: docs.claude.com/en/docs/claude-code.

Working professionally with Claude Code

Write a CLAUDE.md. A file in your repo root that Claude Code reads automatically — your Project instructions, but for code: build commands, conventions, architecture notes, "never touch X" warnings. Ten minutes writing it pays back on every session.

Plan before executing. For non-trivial tasks, ask for a plan first and approve it before edits begin. Steering a plan is cheap; unwinding a wrong implementation is not.

Scope tasks like tickets. "Fix the timezone bug in report generation" succeeds. "Make the app better" wanders. If you couldn't hand it to a junior dev as written, don't hand it to the agent.

Review diffs like a reviewer, not a spectator. You remain the engineer of record. Agentic speed plus rubber-stamp review is how subtle bugs ship. Tests, permissions on risky commands, and git are your safety rails — commit before big agent sessions.

Know the escalation ladder. Quick question → chat. Single-file snippet → chat + Artifact. Multi-file feature, refactor, test-suite work → Claude Code. Choosing the right rung is the skill.

A realistic session

you:    (in repo) claude
you:    Our /export endpoint times out on large datasets.
        Find the cause and propose a fix — plan first.
claude: [reads code] The endpoint loads all rows into memory...
        Plan: stream results, add pagination, update 2 tests. Proceed?
you:    Proceed.
claude: [edits 3 files, runs tests, one fails, fixes it]
        Done — diff ready for review.

That's the shape of modern development: you supply judgment and intent; the agent supplies throughput.

Exercise

Non-developers: in chat, build a small web page in an Artifact, then ask Claude to explain every block and quiz you on it. Developers: install Claude Code on a side project, write a minimal CLAUDE.md, and delegate one well-scoped ticket — plan-first. Review the diff critically. Form your own opinion of where the ceiling is.

Next up

Part 8, the finale: the API, automation, and the habits that make all of this durable — the full professional's toolkit.

Projects: persistent context done right

A Project bundles three things for an ongoing area of work:

1. Knowledge — files and notes uploaded once, visible to every chat inside the Project

2. Custom instructions — standing rules for how Claude behaves there

3. Grouped chats — everything related, in one place

The payoff compounds: instead of re-explaining your situation in every conversation, the context is ambient. A "Job Search" project holds your CV and target roles; every new chat starts already briefed. Say only "cover letter for this posting: [paste]".

Engineering your Project instructions

This is where most people underperform. Instructions are a system prompt you control — treat them with Part 3-level care. A professional template:

ROLE: You are my [editor / analyst / senior dev reviewing my code].
CONTEXT: [2–3 lines: who I am, what this project is]
ALWAYS: [output format, tone, language level, length defaults]
NEVER: [emojis, filler praise, unexplained jargon]
WHEN UNSURE: Ask me one clarifying question instead of guessing.

Rules of thumb, learned the hard way:

• Short beats long. 10 sharp lines outperform two rambling pages — every line is context spent on every message.

• Knowledge is for facts, instructions are for behavior. Your pricing sheet is knowledge; "always quote prices in EUR" is an instruction.

• Stale knowledge poisons silently. An outdated CV in the project means confidently outdated cover letters. Audit your Project files monthly.

Project architectures that work

• The Blog Project — knowledge: style guide, audience description, past top posts. Instructions: voice rules, formatting, SEO checklist. Every draft starts 80% on-brand. (This series is written inside one.)

• The Codebase Companion — knowledge: architecture notes, conventions doc, key schemas. Instructions: "match our patterns; flag breaking changes explicitly."

• The Client Project (one per client) — knowledge: brief, contract scope, correspondence summaries. Instructions: their tone, their terminology. Context-switching between clients becomes instant.

Memory and preferences: the cross-cutting layer

Outside Projects, two account-wide layers travel with you:

• Preferences (Settings) — your global standing instructions, applied everywhere

• Memory — Claude can retain useful details across conversations. Professionals curate it: review what's stored in Settings, delete the noise, keep the durable facts. Treat it like a colleague's notes about you — accurate and current, or corrected.

Precedence intuition: Preferences set your global defaults; Project instructions specialize them; the conversation itself can override both.

Connectors: Claude reaches into your tools

The final layer, and the most quietly transformative. Connectors (built on an open standard called MCP — Model Context Protocol) let Claude securely access other apps you authorize: Google Drive, Gmail, Calendar, Notion, GitHub, Slack, Asana, and a growing directory.

The shift in what a prompt can be:

• "Find our Q3 planning doc in Drive and summarize the open decisions."

• "Check my calendar and draft a reply proposing times I'm actually free."

• "Look at the last 20 GitHub issues and cluster them by root cause."

No copy-paste. Claude reads (and, where you permit, acts) at the source. You enable connectors in Settings and authorize each one explicitly.

Professional caution, stated plainly: a connector that can act (send email, edit tasks) deserves more skepticism than one that only reads. Grant minimal scopes, and review before Claude executes anything irreversible. Power tools, respected.

The complete stack

Notice what you've assembled across six parts:

Preferences (who you are, always) → Project (this domain's knowledge + rules) → Connectors (live data) → Conversation (the task at hand) → Artifact (the deliverable)

That stack — not any single feature — is what "professional Claude usage" actually means.

Exercise

Build one real Project this week using the instructions template above: 3 knowledge files, 8–10 instruction lines. Then connect one read-only connector (Drive or Calendar) and run a prompt that touches both. Note how short your prompts have become — that's infrastructure doing the work.

Next up

Part 7: coding — from chat-assisted programming to Claude Code, the agentic tool that works in your repository like a colleague.

The mental model

The chat is the discussion; the Artifact is the deliverable. Claude creates one automatically for substantial standalone content, or on request: "put this in an artifact." What can live there:

• Documents — reports, guides, long-form drafts

• Code — scripts, components, entire single-file apps

• Interactive apps — real HTML/React that runs live in the panel

• Diagrams — flowcharts, architecture sketches, visualizations

• Tools & games — calculators, quizzes, trackers, prototypes

Iteration is the entire point

The workflow that makes Artifacts powerful:

You: Build a landing page for a freelance photographer. (working page appears) You: Dark theme, add a portfolio grid. (same page, updated) You: The contact button should open a mail link. (updated again)

Claude edits the existing Artifact rather than regenerating it, and you can flip back through versions to compare or recover. Two rules for smooth iteration:

1. One change per message. "Make it dark, add a grid, fix the font, and reorder sections" invites collateral damage. Sequential single changes stay precise.

2. Point at things by name. "In the pricing section, make the middle card highlighted" beats "make it look better."

Building without knowing how to code

Describe outcomes, not technology — Claude picks the stack:

• "A habit tracker where I check off daily habits and see a weekly streak."

• "An interactive quiz: 10 questions on JavaScript basics, explain each wrong answer."

• "A tip-splitting calculator that works well on a phone."

The result runs instantly in the panel. You test it by using it, and request changes in plain language. For many small internal tools, this is legitimately faster than searching for an existing app.

Two power-ups worth knowing:

• Design direction works. "Brutalist," "like a printed newspaper," "playful pastel," "match this screenshot" (attach one — vision from Part 4 applies here too).

• Artifacts can be AI-powered. You can ask for an app that itself talks to Claude — e.g., "build a flashcard app where Claude generates new cards about any topic I type." Apps that think, built by chatting.

Publishing and reuse

From the Artifact panel you can copy, download, or publish — publishing creates a shareable link where others can view and even use interactive artifacts. A quiz for your students, a calculator for your clients, a prototype for your team: shipped from a conversation.

For developers: the honest scope

Artifacts are ideal for prototypes, single-file tools, visualizations, and UI experiments. They are not a full dev environment — no databases, no server-side code, limited external libraries. The professional pattern: prototype the interaction in an Artifact, validate it, then move to a real codebase — with Claude Code (Part 7) doing the heavy lifting there. Prototype in minutes, build properly afterward.

A realistic workflow: the client-proposal machine

1. Chat: discuss the project scope (using Part 3's prompting).

2. "Create an artifact: a structured proposal document from everything above."

3. Iterate: tighten the timeline section, add a pricing table.

4. "Now create a second artifact: a one-page interactive summary I can publish and send as a link."

Deliverable and delivery mechanism, one conversation.

Exercise

Build something real for yourself this week — a unit converter you actually need, a study quiz, a reading tracker. Then request three iterations and publish it. The moment you send your own link to someone is the moment Artifacts click.

Next up

Part 6: the professional operating system — Projects, memory, and connectors that plug Claude into your actual tools.

Grounding: why files change everything

When Claude answers from training data, it's recalling — and recall can be wrong. When Claude answers from a document in its context window, it's reading. Answers become checkable against a source you both share. Professionals exploit this constantly:

Instead of "What does GDPR say about data retention?" (recall, hallucination-prone) upload the actual regulation and ask "According to this document, what are the data-retention rules? Quote the relevant sections." (reading, verifiable)

The instruction "answer only from the document, and say if it's not covered" is one of the most valuable sentences in this series.

Documents: the core workflows

Drag any file into the chat — PDFs, Word docs, text, code. The moves that matter:

• Targeted extraction: "List every deadline, penalty, and obligation in this contract as a table."

• Guided summary: "Summarize this 60-page report for someone deciding whether to fund the project." A summary's audience changes what belongs in it — always state one.

• Interrogation: "What does this lease say about early termination? Quote it." Follow-ups are free; the file stays in context.

• Cross-document work: upload two versions and ask "What changed between these, and which changes shift risk to me?" Multi-file comparison is something humans are slow at and Claude is fast at.

• Transformation: meeting notes → action-item email; research paper → blog outline; requirements doc → test checklist.

Vision: Claude can genuinely see

Uploaded images aren't decorations — Claude analyzes them:

• Screenshot of an error → "Diagnose and fix." (Often faster than copying the text.)

• Whiteboard photo → "Turn this into structured notes with action items."

• A chart → "What's the trend, and what's misleading about how this is presented?"

• UI screenshot → "Critique this design for usability."

• Handwriting, foreign-language forms → transcribe, translate, explain each field.

One built-in boundary: Claude won't identify real people in photos.

The professional layer: executed data analysis

Here's the distinction that separates casual use from real analysis. From Part 1: Claude's mental arithmetic is unreliable. The fix is built in — the analysis tool / code execution (enable it in Settings if needed). When it's relevant, Claude writes actual code, runs it on your file, and reports the computed results.

Upload a CSV and try:

• "Compute monthly revenue growth and flag anomalies. Use code, show your work."

• "Clean this data: trim whitespace, standardize dates, list duplicates — then give me the cleaned file."

• "Which two columns correlate most strongly? Plot it."

The output is the difference between "revenue seems to grow around 10%" (estimated, maybe hallucinated) and "revenue grew 11.4% month-over-month" (computed). When numbers matter, the magic phrase is "use code to calculate this."

Knowing the limits

1. Size ceilings exist. Very large files can exceed the context window. Strategies: split by chapter, or summarize sections in separate chats and synthesize the summaries (context distillation again, from Part 2).

2. Long-document attention. In huge documents, details buried mid-file get less attention than the start and end. For contracts and compliance work, ask section-targeted questions rather than one giant "check everything."

3. Scanned PDFs vary. Claude handles them, but low-quality scans degrade extraction — spot-check quotes against the page.

4. Verification discipline stays. Grounding shrinks hallucination; it doesn't abolish it. For consequential documents, verify quoted passages exist. The habit costs a minute.

A realistic workflow: competitor research

1. Upload three competitor pricing PDFs → "Extract all plans and prices into one comparison table."

2. "Use code: compute the average price per feature tier."

3. "Which competitor changed positioning compared to this older PDF?" (upload it)

4. End with distillation: "Summarize the strategic picture in 10 bullets" → carry that into a fresh chat for strategy work.

Four prompts. That used to be an afternoon.

Exercise

Take a real spreadsheet or export (bank statement, sales data, anything). Ask one question without code, then the same question with "use code to calculate this." Compare the answers — this lesson sticks best when you see the difference yourself.

Next up

Part 5: Artifacts — turning conversations into working documents, tools, and apps you can actually ship.

Level 1: The foundation formula

Every solid prompt covers four things:

Role + Task + Context + Format

❌ "Write a blog post about fitness."

✅ "You're a fitness coach who writes for busy professionals (role). Write a blog post about 10-minute morning workouts (task). Audience: office workers with no equipment, mostly beginners (context). 600 words, friendly tone, numbered exercise list (format)."

Specificity isn't decoration — every constraint eliminates thousands of wrong answers. If you learn nothing else, learn this formula. Now let's go further.

Level 2: Intermediate techniques

Few-shot prompting: show, don't describe

The single biggest upgrade from Level 1. Instead of describing the style you want, show 1–3 examples:

Rewrite product names in our brand voice. Examples:

"Steel water bottle 750ml" → "The Everyday Bottle — built for your whole day"
"Wireless mouse M2" → "The Quiet Click — work without the noise"

Now rewrite: "Ergonomic office chair X200"

Claude is exceptional at pattern-matching. Two good examples routinely outperform two paragraphs of instructions.

Structure with delimiters

When prompts mix instructions with data (a document, an email, code), separate them clearly. Claude is specifically trained to respect XML-style tags:

Summarize the customer complaint below in 3 bullets,
then classify severity as low/medium/high.

<complaint>
[paste the full text here]
</complaint>

Now "summarize" can never bleed into the complaint text, and long inputs stay unambiguous. This habit matters more the bigger your inputs get.

Positive instructions beat negative ones

"Don't be formal" forces Claude to guess what you do want. "Write like you're texting a smart friend" nails it. Say what to do, not only what to avoid.

Constrain the output shape

"Respond as a markdown table with columns X, Y, Z", "exactly 5 options, one line each", "return only valid JSON, no explanation". If you'll reuse the output somewhere (a spreadsheet, code, your blog), specify the shape and skip the cleanup.

Level 3: Advanced techniques

Make Claude think before answering

For analysis, evaluation, or anything with a judgment call:

"Before giving your recommendation, first reason through the pros and cons of each option step by step. Then give your final answer with a confidence level."

Forcing visible intermediate reasoning measurably improves final answers — and lets you audit where the reasoning went wrong when it does. (For truly hard problems, combine this with the extended-thinking toggle from Part 2.)

Prompt the reversal

Ask Claude to argue against itself: "Now make the strongest case against your own recommendation." This is the fastest hallucination- and blind-spot-check that exists, and professionals use it constantly for decisions that matter.

Meta-prompting: recursion that actually works

Claude is excellent at improving prompts — including yours:

"I want Claude to generate weekly LinkedIn posts from my rough notes. Write the ideal reusable prompt for this. Ask me clarifying questions first."

Answer its questions, get a polished prompt, save it as a template. You've just automated your own prompt engineering.

Build a prompt library

The final professional habit: stop rewriting prompts you've already perfected. Keep a note file (or, better, Project instructions — Part 6) of your proven templates: the report summarizer, the code reviewer, the email drafter. Over months, this library becomes a genuine productivity asset.

Debugging a bad answer: a checklist

When output disappoints, check in order:

1. Ambiguity — could my request mean two things? (Most common cause.)

2. Missing context — does Claude know what I know?

3. No example — would one sample fix the style?

4. Contaminated conversation — is an earlier wrong turn still in context? → edit the message (Part 2), don't pile on corrections

5. Wrong model / thinking off — hard problem on a fast setting?

Exercise

Take a task you do weekly. Write a Level 1 prompt, then upgrade it with one example and XML structure, then run the meta-prompting technique on it. Compare all three outputs — and save the winner to your new prompt library.

Next up

Part 4: feeding Claude real material — documents, images, spreadsheets — and running genuine data analysis with executed code.

The two-minute setup

Go to claude.ai, sign up with email, Google, or Apple. The free plan is permanent, no card required. Install the mobile app and, if you work at a computer all day, the desktop app — a global keyboard shortcut to summon Claude changes how often you actually use it.

Interface essentials, quickly: the message box, the sidebar (past chats, searchable), the model picker dropdown, and the attach button for files. Done. Now the part that matters.

Mechanic 1: Every message replays the whole conversation

From Part 1: Claude has no hidden memory — the conversation transcript is its memory, re-read on every turn. Three consequences:

1. Long chats degrade. Responses slow down, usage burns faster, and details from 50 messages ago get less attention.

2. Everything in the chat influences everything after. A wrong assumption Claude made early keeps echoing unless you correct it explicitly.

3. A fresh chat is a reset button — often the strongest debugging move you have.

Mechanic 2: When to continue vs. when to restart

Continue the same chat when you're iterating on the same task and the accumulated context helps ("now make section 2 more formal").

Start a new chat when:

• You're switching topics — even related ones

• The conversation went sideways and corrections aren't sticking

• You've finished a phase (research done → new chat for writing, pasting in only the summary)

That last one is a professional pattern called context distillation: end a long session with "Summarize everything we decided as a brief I can paste into a new conversation" — then start clean with just that brief. Best of both worlds: full memory of decisions, none of the clutter.

Mechanic 3: Edit, don't pile up

Most people respond to a bad answer by adding another message: "no, I meant..." — which leaves the misunderstanding in the context, still influencing everything.

The stronger move: edit your original message (hover over it) and rephrase. The conversation re-branches from that point as if the bad turn never happened. You can also retry a Claude response to get a different take. Cleaning history beats correcting it.

Mechanic 4: Choose models deliberately

The model picker isn't decoration:

• Fast models (Haiku/Sonnet) for drafts, summaries, simple questions — quicker and lighter on your usage limits

• Top models (Opus and above) for complex reasoning, tricky code, high-stakes writing

You can switch mid-conversation — the new model sees the full history. Pattern: explore cheap, finish strong.

Also try extended thinking (a toggle, on supported models) for genuinely hard problems — Claude reasons step-by-step before answering. Slower, but noticeably better on math, logic, and planning. Leave it off for casual questions.

Mechanic 5: Configure once, benefit forever

Five minutes in Settings pays off permanently:

• Preferences — standing instructions applied to every chat: "I'm a non-native English speaker, keep language natural and simple. Never use emojis. When I ask for code, always include comments."

• Styles — switch response style (concise / explanatory / formal) per task, or create your own from writing samples

• Memory — lets Claude carry useful details across conversations; review and edit what's stored in Settings

• Feature toggles — make sure web search and the analysis tool are enabled; you'll need both later in this series

A worked example

❌ Novice: one 80-message chat mixing a trip plan, a CV review, and a Python bug — by the end, Claude is confusing the contexts.

✅ Practitioner: three chats. The trip chat ends with "summarize the itinerary decisions"; that summary starts a clean booking-research chat, on a fast model, with web search on.

Same tool. Completely different results.

Exercise

1. Set up your Preferences (3 lines minimum).

2. Start a chat on any topic, then deliberately edit an earlier message and watch the conversation re-branch.

3. End a chat with the context-distillation prompt and carry the summary into a new one.

Next up

Part 3: prompting — from the basic formula to the structured, example-driven techniques professionals use.

What is Claude?

Claude is an AI assistant built by Anthropic, an AI safety company. On the surface, it's a chat interface: you write, it answers. Underneath, it's a large language model (LLM) — a neural network trained on enormous amounts of text to predict what comes next in a sequence.

That "predict the next word" framing sounds trivial, but at sufficient scale it produces something remarkable: a system that can reason through problems, write working code, analyze documents, and explain almost anything. You don't need the math — but three concepts will make you dramatically better at using it.

Concept 1: Tokens — how Claude reads

Claude doesn't see letters or words. Text is broken into tokens — chunks of roughly 3–4 English characters. "Understanding" might be two tokens; "cat" is one.

Why should you care?

• Limits are measured in tokens. Message limits, document sizes, response lengths — all token-based.

• Cost (on the API) is per token. Verbose prompts literally cost more.

• It explains quirks. Counting letters in a word is oddly hard for LLMs because they never see individual letters.

Concept 2: The context window — Claude's working memory

Everything Claude can "see" during a conversation — your messages, its replies, uploaded files, instructions — lives in the context window. Claude's is huge (hundreds of pages of text), which is one of its signature strengths.

Two professional-grade implications:

1. Claude re-reads the entire conversation on every message. Nothing is "remembered" between turns in some hidden brain — the memory is the transcript. This is why long chats slow down and consume usage limits faster.

2. When context fills up, quality degrades before it fails. Details from early in a very long conversation get less attention. Pros start fresh chats deliberately and re-supply only the relevant context.

We'll turn this into concrete conversation-management tactics in Part 2.

Concept 3: Training cutoff vs. tools

Claude's base knowledge stops at its training cutoff date. It doesn't inherently know today's news, prices, or your company's data. What closes the gap is tools: web search, file uploads, code execution, and connectors to apps like Google Drive. A large part of professional Claude usage is knowing which knowledge lives in the model and which must be brought in via tools. This series covers all of them.

The model family

Claude ships in several models — a speed/capability trade-off:

• Haiku — fastest and cheapest; ideal for simple, high-volume tasks

• Sonnet — the balanced workhorse for everyday work

• Opus and above — maximum capability for complex reasoning, hard coding problems, and long documents

You can switch models mid-conversation from a dropdown. A practical pattern you'll use later: draft and explore with a fast model, then switch up for the hard final step.

What Claude is genuinely good and bad at

Strong: writing and editing, coding, summarizing and analyzing long documents, structured reasoning, translation with natural phrasing, brainstorming, explaining at any level.

Weak — know these cold:

• Hallucination. Claude can state false things fluently and confidently — especially niche facts, statistics, citations, and URLs. Verification is your job; we'll build it into every workflow in this series.

• Arithmetic at scale. For real calculations, have Claude write and run code (Part 4) instead of computing "in its head."

• Real-time anything — without web search enabled.

The one-sentence summary

Claude is a reasoning engine with a large but finite working memory, frozen knowledge, and pluggable tools — and your skill is in what you put into that memory.

Every technique in the next seven parts is an application of that sentence.

Exercise

Ask Claude: "Explain how large language models work, then explain what tokens and context windows are, using an analogy from cooking." Then ask: "What are your own limitations I should watch out for?" — Claude is refreshingly honest about itself.

Next up

Part 2: setting up properly and mastering conversation mechanics — model switching, context hygiene, and the settings most users never find.

Two battle-tested strategies exist to defend against this:

Idempotency and Nonce. They look similar on the surface, but they serve different purposes. Let's break them down.

The Problem: Why Do Duplicates Happen?

Before diving into solutions, let's understand the enemy.

Client  ──── POST /pay ────►  Server
Client  ◄──── timeout ──────  Server (processing...)
Client  ──── POST /pay ────►  Server  ← 💣 DUPLICATE!

The client never got a response, so it retried — but the first request was already processed. Now you have two charges.

This happens in:

• Mobile apps on unstable connections

• Message queues (at-least-once delivery)

• Retry logic in service-to-service communication

• Frontend double-clicks

Solution 1: Idempotency

What Is It?

An operation is idempotent if calling it once or calling it many times produces the same result.

f(f(x)) = f(x)

Think of a light switch that only turns on: pressing it once or ten times — the light is on. Same result.

In HTTP terms: GET, PUT, and DELETE are naturally idempotent. POST is not — that's where you need to enforce it yourself.

How It Works

The client generates a unique Idempotency Key and sends it with every request. The server stores the result of the first execution. On any subsequent request with the same key, the server returns the cached result without re-executing the operation.

Request 1: POST /pay  {idempotency-key: "abc-123"}  → Processed ✅  (result saved)
Request 2: POST /pay  {idempotency-key: "abc-123"}  → Returned from cache ✅  (not re-executed)
Request 3: POST /pay  {idempotency-key: "abc-123"}  → Returned from cache ✅  (not re-executed)

Important Notes

• The key should be generated by the client (usually a UUID v4).

• Store idempotency records in a persistent store (DB or Redis), not in memory.

• Set a TTL on records (e.g., 24 hours or 7 days) — you don't need them forever.

• Handle concurrent requests with the same key using a database unique constraint or distributed lock.

Solution 2: Nonce

What Is It?

Nonce stands for "Number used Once". It's a unique token that is valid for a single use only. Once consumed, it's permanently invalidated — even if the operation behind it failed.

The key difference from idempotency: a nonce doesn't cache results. It just asks: "Has this token been used before?"

How It Works

Request 1: POST /transfer  {nonce: "xyz-789"}  → Valid, consumed ✅
Request 2: POST /transfer  {nonce: "xyz-789"}  → REJECTED ❌ (already used)

Even if request 1 failed for some business reason, request 2 with the same nonce is still rejected. The client must generate a new nonce for a new attempt.

Important Notes

• You can use Redis SETNX (SET if Not eXists) for atomic check-and-consume. Never do a separate GET then SET — that's a race condition.

• Set a reasonable TTL based on your use case (5–15 minutes for form submissions, longer for payment flows).

• The client must generate a new nonce for every new attempt. Nonces are not reusable.

• Nonces work great for one-time form submissions, OTP verification, and CSRF protection.

Side-by-Side Comparison

When to Use Which?

Is the client allowed to retry with the same intent?
    │
    ├── YES → Use Idempotency
    │         (same operation, same result expected)
    │         Example: "Charge $100 for order #456"
    │
    └── NO  → Use Nonce
              (one shot only, new attempt = new token)
              Example: "Submit this form", "Verify this OTP"

Real-world rule of thumb:

• Payment APIs → Idempotency (Stripe, PayPal both use this)

• Authentication flows, form submissions → Nonce

• CSRF protection → Nonce

• Webhook delivery → Idempotency

Combining Both

In high-security systems, you can use both together. This gives you the best of both worlds: replay protection in the short term (nonce) and safe retries over time (idempotency).

Summary

Both patterns are essential tools in any production backend. Idempotency is your friend when you need fault-tolerant retry logic. Nonce is your guard against replay attacks and accidental double submissions.

Once you start thinking in these terms, you'll spot the need for them everywhere.

I've lived this more than once. When everything is connected, the hardest part isn't writing code. It's building a mental model of how the pieces fit together before you dare to touch anything.

Most "code graph" tools don't help here. They render a hairball of nodes and edges and proudly tell you the codebase has 1,400 files. Great — now what?

A Graph That Teaches

Understand Anything is an open-source plugin that turns any codebase into an interactive knowledge graph you can actually explore and learn from. The pitch that sold me is right there on the homepage: other tools show you structure, this one shows you meaning — how your code maps to real business domains, processes, and flows.

It runs as a plugin for Claude Code, but also works with Codex, Cursor, Copilot, Gemini CLI, OpenCode, and a long list of others. Under the hood it analyzes your project with a multi-agent pipeline, builds a graph of every file, function, class, and dependency, then hands you a dashboard to explore it all visually.

How It Actually Works

The clever part is the hybrid approach. It doesn't rely purely on an LLM guessing about your code, and it doesn't rely purely on a parser that can't understand intent. It uses both:

• Tree-sitter (deterministic) parses your source into a syntax tree and extracts the hard structural facts — imports, exports, function and class definitions, call sites, inheritance. Same input, same output, every run. This is what makes the structural graph reproducible.

• An LLM (semantic) then reads that parsed structure alongside the original source to produce what a parser never can — plain-English summaries, architectural layer assignments, and business-domain mapping. The why, not just the what.

That split is the whole point. The structural edges are stable and trustworthy, while the semantic layer captures what a file is actually for.

The /understand command orchestrates a handful of specialized agents — a scanner to discover files and detect frameworks, a file analyzer to extract functions and build nodes and edges, an architecture analyzer to identify layers, a tour builder, and a reviewer to validate the graph. There's also a domain analyzer that extracts business domains, flows, and process steps. File analyzers run in parallel and the whole thing supports incremental updates, so re-running it only re-analyzes the files that changed.

Get Started in 30 Seconds

If you're already using Claude Code, installation is two lines:

/plugin marketplace add Egonex-AI/Understand-Anything
/plugin install understand-anything

Then point it at your project:

/understand

The pipeline scans everything and saves a knowledge graph to .understand-anything/knowledge-graph.json. Open the dashboard with:

/understand-dashboard

You get an interactive web view of your codebase as a graph — color-coded by architectural layer, searchable, and clickable. Select any node and you see its code, its relationships, and a plain-English explanation of what it does.

On a non-Claude platform, there's a one-line installer instead:

curl -fsSL https://raw.githubusercontent.com/Egonex-AI/Understand-Anything/main/install.sh | bash

The Commands I'd Actually Use Day to Day

Beyond the initial scan, a few commands map directly onto real situations I run into:

• Ask anything about the codebase

  • /understand-chat How does the payment flow work?

• See what your current changes will affect — before you commit

  • /understand-diff

• Deep-dive into one specific file or function

  • /understand-explain src/auth/login.ts

• Generate an onboarding guide for a new teammate

  • /understand-onboard

• Extract business domains, flows, and steps

  • /understand-domain

That /understand-diff is the one that caught my attention. Knowing the ripple effect of a change before you commit — which services and flows it touches — is exactly the kind of thing that's painful to reason about manually in a tightly coupled microservice setup.

Sharing the Graph With Your Team

Here's the detail I appreciated most as someone who's written more than one internal onboarding post. The graph is just JSON, so you can commit it once and let teammates skip the whole pipeline. Anyone who clones the repo gets the map for free — ideal for onboarding, PR reviews, and docs-as-code.

You commit everything in .understand-anything/ except the local scratch files, and there's an --auto-update flag that installs a post-commit hook so the graph stays in sync with the code on every commit. No more "the diagram is three months out of date."

Why I Think It's Worth Trying

The framing on the project sums it up better than I can: the goal isn't a graph that impresses you with how complex your codebase is — it's a graph that quietly teaches you how every piece fits together.

For anyone who regularly inherits unfamiliar code, mentors new hires, or just wants a faster way to build a mental model of a system, this is a genuinely useful tool. It's MIT-licensed and open source, so there's no cost to giving it a spin on your own repo.

If you want to see it before installing anything, there's a live demo you can pan, zoom, and search right in the browser. Otherwise, point it at your messiest service and see what it teaches you.

Have you tried it on a large codebase? I'd be curious how it holds up on a multi-service setup — let me know in the comments.

Why do we choose redisson?

While multiple Redis clients are available for Java, including Jedis, Lettuce, and Spring Data Redis, Redisson stands out due to its rich feature set, ease of use, and strong support for distributed systems. New: Redisson now supports valkey.

Clients Comparison

Below is a quick comparison of the most popular Java Redis clients:

Based on this table and references, Redisson is a great choice.

References

• Feature Comparison: Redisson vs Spring Data Redis

• Feature Comparison: Redisson vs Lettuce

• Feature Comparison: Redisson vs Jedis

Spring Boot Starter

In this starter, you'll learn how to use Redis with the Redisson library. All examples use Spring Boot 3.4.5 with Redisson 3.45.1.

ATTENTION: You can see full examples in this GitHub repo.

To get started, include the following dependency:

<dependency>
   <groupId>org.redisson</groupId>
   <artifactId>redisson-spring-boot-starter</artifactId>
   <version>3.45.1</version>
</dependency>

While you can also use the redisson artifact directly, using the redisson-spring-boot-starter is recommended. It provides useful Spring Boot auto-configurations, health checks through Actuator, and additional Spring-specific integrations.

We will configure the Redis cluster based on the doc we provided before.

spring:
  redis:
    redisson:
      file: classpath:redisson.yaml

In redisson.yaml:

clusterServersConfig:

There are a lot more important configs that you can set in this file that you can see in Appendix 1. Tuning these configs is based on your needs.

Important Notes

• Connection Pool Size

  • Ensure that the connection pool size is properly configured based on your workload. A pool that's too small can lead to connection timeouts under high load, while an oversized pool may waste resources.

• Cluster Scan Frequency

  • Avoid setting the scanInterval too low. Scanning the cluster too frequently can create unnecessary network overhead and impact performance, especially in large-scale deployments.

• Ping Interval Configuration

  • Frequent pings to maintain connections can lead to excessive traffic and latency. Tune the pingConnectionInterval based on your environment and latency tolerance.

References

• Redisson Configuration

Redis Health for Spring Boot App

Spring Boot Actuator provides a default Redis health indicator, but it offers very limited information. By default, it returns something like:

"redis": {
    "status": "UP",
    "details": {
        "cluster_size": 3,
        "slots_up": 16384,
        "slots_fail": 0
    }
}

To get more detailed insights—such as the health status of Redis slave nodes—you can implement a custom health indicator based on your needs. For example:

@Slf4j
@Component
@RequiredArgsConstructor
@FieldDefaults(level = AccessLevel.PRIVATE, makeFinal = true)
public class RedisMastersHealthIndicator implements HealthIndicator {
    RedissonClient redissonClient;

    @Override
    public Health health() {
        RedisCluster cluster = redissonClient.getRedisNodes(RedisNodes.CLUSTER);
        Collection<RedisClusterMaster> masters = cluster.getMasters();
        for (RedisClusterMaster master : masters) {
            try {
                if (!master.ping()) {
                    log.error("Redis slave node {} is DOWN", master);
                    return Health.down().withDetail("failedNode", master.toString()).build();
                }
            } catch (Exception e) {
                log.error("Redis slave node {} is DOWN", master, e);
                return Health.down().withDetail("failedNode", master.toString()).build();
            }
        }
        return Health.up().withDetail("mastersChecked", masters.size()).build();
    }
}

This custom indicator checks the health of Redis slave nodes and reports if any of them are unreachable.

Distributed Collections (Data Structures)

These are redis data structures with redisson wrapper:

Example of each:

RBucket<String> bucket = redissonClient.getBucket("myKey");
bucket.set("Hello, Redisson!");
System.out.println(bucket.get());

RList<String> list = redissonClient.getList("myList");
list.add("Apple");
list.add("Banana");
System.out.println(list.get(0));

RSet<String> set = redissonClient.getSet("mySet");
set.add("A");
set.add("B");
set.add("A");
System.out.println(set.contains("B"));

RMap<String, Integer> map = redissonClient.getMap("myMap");
map.put("views", 10);
System.out.println(map.get("views"));

RScoredSortedSet<String> scoredSet = redissonClient.getScoredSortedSet("mySortedSet");
scoredSet.add(9.5, "Alice");
scoredSet.add(8.0, "Bob");
System.out.println(scoredSet.first());

RGeo<String> geo = redissonClient.getGeo("myGeo");
geo.add(13.361389, 38.115556, "Palermo");
geo.add(15.087269, 37.502669, "Catania");
System.out.println(geo.dist("Palermo", "Catania", GeoUnit.KILOMETERS));

RBitSet bitSet = redissonClient.getBitSet("myBitSet");
bitSet.set(0, true);
bitSet.set(1, false);
System.out.println(bitSet.get(0));

Map & Set

Redisson provides various Map structure implementations with multiple important features (lots of them are only for redisson pro 🙁): (In appendix 2, you can read about the meaning of local cache and data partitioning)

1. No eviction

   1. Available implementations:

      1. getMap()

      2. getLocalCachedMap();

2. Scripted eviction

   1. Allows for defining time to live or max idle time parameters per map entry.

   2. Eviction is done on the redisson side through a custom-scheduled task that removes expired entries using La ua script.

   3. Available implementations:

      1. getMapCache()

         1. map.putIfAbsent("key2", new SomeObject(), 40, TimeUnit.SECONDS, 10, TimeUnit.SECONDS);

3. Advanced eviction

   1. All for Redisson Pro

4. Native eviction

   1. Allows for defining time-to-live parameters per map entry.

   2. Doesn't use an entry eviction task, entries are cleaned on Redis side.

   3. Requires Redis 7.4+.

   4. Available implementations:

      1. getMapCacheNative()

Similar methods are available for RSet as well. For more details, refer to the documentation.

Danger: Don't Rely on Redisson TTL Alone. If TTL is critical to your business logic (e.g. session expiry, rate limiting, locks):

• Always use Redis-native expiration.

• Avoid relying on Redisson TTL if app uptime isn't guaranteed.

Solutions may:

• RBucket and RMapCacheNative support redis ttl

• Sometimes you can use the Redis key TTL instead of per-entry TTL.

  • redissonClient.getKeys().expire("myMap", 10, TimeUnit.SECONDS);

Multimap

Multimap for Java allows binding multiple values per key. This object is thread-safe. Keys are limited to 4,294,967,295 elements.

Multimap distributed object for Java with eviction support implemented by separated MultimapCache object. There are RSetMultimapCache and RListMultimapCache objects for Set and List based Multimaps respectively. Eviction task is started once per unique object name at the moment of getting a Multimap instance.

RSetMultimap<Integer, Integer> set = redissonClient.getSetMultimap("mySetMultimap");
set.put(1, 1);
set.put(1, 2);
set.put(1, 3);
set.put(1, 3);
set.put(2, 1);
System.out.println("SetMultimap:");
set.keySet().forEach(key -> {
    System.out.println(key + " => " + set.get(key));
});

RListMultimap<Integer, Integer> map = redissonClient.getListMultimap("myListMultimap");
map.put(1, 1);
map.put(1, 2);
map.put(1, 3);
map.put(1, 3);
map.put(2, 1);
System.out.println("ListMultimap:");
map.keySet().forEach(key -> {
    System.out.println(key + " => " + map.get(key));
});

Danger: Redisson's RSetMultimap and RListMultimap can create many Redis keys, especially when you have many keys or values.

When you use:

RSetMultimap<Integer, Integer> map = redissonClient.getSetMultimap("mySetMultimap");

Redisson internally creates one Redis key for each map entry (and some metadata), like:

mySetMultimap:{key}

mySetMultimap, mySetMultimap:{1}, mySetMultimap:{2}, and …

This causes many problems, such as key growth if there are many keys/values, Slower operations for listing or scanning, and …

These are some solution for this:

• Use RMap<K, Set> instead of Redisson Multimap.

• Distribute keys using modulo: multimap:.

• Use RMap<String, String> with CSV/JSON values (Not Recommend)

Important Notes

• Redisson allows you to bind listeners to certain collections, such as RMap. For more details, refer to the documentation.

• Redisson also supports distributed queues.

• Redisson also supports time series.

• For some data structures choosing a good codec will save a lot of cost.

References

• Distributed Objects

• Distributed Collections

Distributed Locks

Redisson provides robust distributed locking mechanisms to ensure safe concurrent access in clustered or multi-pod environments.

You can acquire a lock using the following methods:

• RLock.lock()

  • Blocks indefinitely until the lock is acquired.

  • Simple, but use it only when you're sure the lock will be released eventually.

• RLock.tryLock(waitTime, leaseTime, timeUnit)

  • Attempts to acquire the lock:

    • Waits up to waitTime.

    • If acquired, holds for lease time and then auto-releases.

  • Safer than lock() in most distributed environments.

• RReadWriteLock

  • Allows many readers OR one writer.

  • Useful when:

    • Multiple pods need concurrent read access.

    • Only one pod should perform write/update at a time.

• RFairLock

  • Ensures first-come, first-served access to the lock.

  • Use when fairness (e.g., request order) is important.

• RMultiLock

  • Combines multiple locks into a single logical lock.

  • Acquires all or none. Ideal when working with multiple related resources (e.g., account transfers).

Important Notes

• Lock key should be unique per logical task (e.g. "lock:register:" + userId).

• Always release locks in finally blocks to prevent deadlocks.

• Avoid overusing locks (they add latency and reduce throughput)

  • Use locks only when multiple related keys or complex logic must be executed as a unit.

  • Favor atomic Redis operations when possible.

    • Atomic operations like INCR, SETNX, and HINCRBY are built-in, single-step commands in Redis.

    • They avoid the overhead of acquiring/releasing distributed locks.

• By default lock watchdog timeout is 30 seconds and can be changed through Config.lockWatchdogTimeout setting.

• Read full locks here based on your needs.

References

• Distributed locks and synchronizers

Distributed Tasks

Redisson supports distributed task execution using the RExecutorService, allowing tasks to be submitted and executed across multiple nodes in a cluster.

Here's a compact summary of the most useful Redisson Distributed Task methods:

• submit(task)

  • Submits a task for execution on any available node.

• schedule(task, delay, unit)

  • Schedules a one-time task to run after a delay.

Important Notes

• registerWorkers is crucial when using Redisson's RScheduledExecutorService or RExecutorService.

  • It registers the local JVM process as a worker node that can execute submitted tasks.

  • executor.registerWorkers(...) tells Redisson: “This instance is available to run background tasks.”

  • WorkerOptions.defaults().workers(1) means: “Allow 1 background thread in this instance to execute submitted jobs.”

• You can add listeners like TaskSuccessListener to options.

• You can add runnable and callable.

• You can use cron in the schedule method.

References

• Executor service

Pub/Sub

Java RTopic object implements Publish / Subscribe mechanism based on Redis Pub/Sub. It allows clients to subscribe to events published with multiple instances of RTopic objects with the same name. Listeners are re-subscribed automatically after reconnection or failover.

If you're not using Reliable Topic, any messages published while the client is disconnected will be lost. Use RReliableTopic if message durability is critical.

References

• Distributed publish/subscribe

Streams

Redisson Streams provides a Redis-based implementation of the Redis Streams data structure, which is ideal for building scalable, event-driven systems.

Important Notes

• Always ack() messages after successful processing to remove them from the pending list.

• Redisson has more methods like pending and claim that you can use based on your needs.

References

• Stream

General Important Notes

Before using Redis in your service, please review and follow these important guidelines:

1. **Evaluate if Redis is the right choice:
   **Make sure Redis fits your architecture and use case. Redis is best suited for high-performance caching, real-time analytics, pub/sub messaging, distributed locks, and short-lived data. It is not ideal for storing large or critical persistent data unless configured with proper persistence and backup strategies.

2. **Design for fault tolerance:
   **Your service must handle Redis failures gracefully. Redis is an external dependency and can become temporarily unavailable.

   • Always implement fallback mechanisms or default behaviors when Redis is down.

   • Avoid making your entire service dependent on Redis availability.

3. **Be careful with data eviction and TTLs:
   **If you use Redis as a cache, always define appropriate TTL values to prevent stale data and memory exhaustion.

   • Do not assume Redis data will persist indefinitely.

4. **Use proper serialization and data structures:
   **Choose an efficient serialization format and be consistent across services.
   Use the right Redis data type for your purpose (String, Hash, List, Set, Sorted Set, etc.)

5. **Monitor and observe Redis usage:
   **Continuously monitor key metrics like memory usage, hit/miss ratio, latency, and connection counts.
   Tools like RedisInsight and Grafana metrics can help.

Helpful Definitions

Appendix

Appendix 1

Most important configs are:

- clientName:

• Default value: null

• Name of client connection.

- nodeAddresses:

• Redisson automatically discovers the cluster topology.

- readMode:

• Default value: SLAVE

• Set node type used for read operation.

• Available values: SLAVE, MASTER and MASTER_SLAVE

- subscriptionMode:

• Default value: MASTER

• Set node type used for pub/sub operation.

• Available values: SLAVE and MASTER

- scanInterval:

• Default value: 1000

• Applied clusters topology scans.

- pingConnectionInterval:

• Default value: 30000

• This setting allows for detecting and reconnecting broken connections, using the PING command.

• Set to 0 to disable.

- slave/master/subscriptionConnectionMinimumIdleSize:

• Default value: 24

• Minimum idle connections amount per node/channels.

- slave/master/subscriptionConnectionPoolSize:

• Default value: 64

• Maximum connection pool size per node/channels.

- connectTimeout:

• Default value: 10000

• Timeout in milliseconds during connecting to server.

- idleConnectionTimeout:

• Default value: 10000

• If a pooled connection is not used for a timeout time and the current connections amount is bigger than the minimum idle connections pool size, then it will be closed and removed from the pool.

- subscriptionTimeout:

• Default value: 7500

• Defines subscription timeout in milliseconds applied per channel subscription.

- timeout:

• Default value: 3000

• Server response timeout in milliseconds.

• Starts countdown after a command is successfully sent.

- retryAttempts:

• Default value: 3

• Error will be thrown if command can’t be sent to server after retryAttempts. But if it is sent successfully then timeout will be started.

- retryInterval:

• Default value: 1500

• Time interval in milliseconds, after which another attempt to send a command will be executed.

- failedSlaveReconnectionInterval:

• Default value: 3000

• Interval of Slave reconnection attempts, when it was excluded from an internal list of available servers.

• On each timeout event, Redisson tries to connect to the disconnected server.

- failedSlaveNodeDetector:

• Default value: org.redisson.client.FailedConnectionDetector

• Defines the failed Slave node detector object which implements failed node detection logic via the org.redisson.client.FailedNodeDetector interface.

• Available implementations:

  • org.redisson.client.FailedConnectionDetector

  • org.redisson.client.FailedCommandsDetector

  • org.redisson.client.FailedCommandsTimeoutDetector

- keepAlive:

• Default value: false

• Enables TCP keepAlive for connection.

- tcpNoDelay:

• Default value: true

• Enables TCP noDelay for connections.

- nettyThreads:

• Default value: 32

• Defines the number of threads shared between all internal clients used by Redisson.

• Netty threads are used for response decoding and command sending.

• 0 = cores_amount * 2

- threads:

• Default value: 16

• Threads are used to execute the listener's logic of the RTopic object, invocation handlers of the RRemoteService, the RTopic object and RExecutorService tasks.

- codec:

• Default value: org.redisson.codec.Kryo5Codec

• Used during read/write data operations.

• Several serialization implementations are available.

- lockWatchdogTimeout:

• Default value: 30000

• This prevents infinity-locked locks.

• This parameter is only used if an RLock object is acquired without the leaseTimeout parameter.

Appendix 2

Here is a meaning of local cache and data partitioning:

• Local cache

  • So called near cache used to speed up read operations and avoid network round trips. It caches Map entries on Redisson side and executes read operations up to 45x faster in comparison with common implementation.

• Data partitioning

  • Although any Map object is cluster compatible its content isn't scaled/partitioned across multiple master nodes in the cluster. Data partitioning allows to scale available memory, read/write operations and entry eviction processes for individual Map instances in a cluster.

To see all data structures, including extension data types, visit the link below:
Redis Data Types Documentation + Cheat Sheets

The link below provides the time complexity for all Redis commands, which is very helpful in selecting the right data structure for your use case:
Redis Command Time Complexity Cheat Sheet

Strings

Redis strings store sequences of bytes, including text, serialized objects, and binary arrays.

Example

> SET bike:1 Deimos
OK
> GET bike:1
"Deimos"

Usage

• Storing simple key-value pairs

• Caching responses

• Counting (using INCR, DECR)

Important Notes

• Most string operations are O(1)

• Be careful with the SUBSTR, GETRANGE, and SETRANGE commands, which can be O(n)

• Redis string can be a maximum of 512 MB (Danger)

Helpful Links

• https://redis.io/docs/latest/develop/data-types/strings

• https://redis.io/docs/latest/commands/?group=string

Lists

Redis lists are ordered collections of strings, implemented as linked lists.

Example

> RPUSH todo:123 "Buy milk"
> RPUSH todo:123 "Walk the dog"
> LPUSH todo:123 "Check emails"
> LRANGE todo:123 0 -1
1) "Check emails"
2) "Buy milk"
3) "Walk the dog"

Usage

• Implementing queues (FIFO, LIFO)

• Twitter social network takes the latest tweets posted by users into Redis lists - reference

Important Notes

• Operations that access its head or tail are O(1)

• Commands that manipulate elements within a list are usually O(n) (Danger)

  • Like LINDEX, LINSERT, and LSET

• Redis streams is an alternative to lists when you need to store and process an indeterminate series of events

Helpful Links

• https://redis.io/docs/latest/develop/data-types/lists

• https://redis.io/docs/latest/commands/?group=list

Sets

Redis Set is an unordered collection of unique strings.

Example

> SADD users:online Alice
> SADD users:online Bob
> SADD users:online Alice
> SMEMBERS users:online
1) "Alice"
2) "Bob"

Usage

• Track unique items (e.g., track all unique IP addresses accessing a given blog post).

• Represent relations (e.g., the set of all users with a given role).

• Perform common set operations such as intersection, unions, and differences.

Important Notes

• Most set operations, including adding, removing, and checking whether an item is a set member, are O(1)

• SMEMBERS command is O(n) and returns the entire set in a single response

Helpful Links

• https://redis.io/docs/latest/develop/data-types/sets

• https://redis.io/docs/latest/commands/?group=set

Hashes

Redis hashes are key-value maps, useful for storing objects.

Example

> HSET user:100 name "Alice" age "30"
> HGET user:100 name
"Alice"
> HGETALL user:100
1) "name"
2) "Alice"
3) "age"
4) "30"

Usage

• Caching database rows and …

Important Notes

• New in Redis Community Edition 7.4 is the ability to specify an expiration time or a time-to-live (TTL) value for individual hash fields.

• Most Redis hash commands are O(1).

• A few commands and most of the expiration-related commands are O(n) (Danger)

  • Like HKEYS, HVALS, HGETALL

• The number of fields in a hash is limited only by the total memory available on the Redis server.

Helpful Links

• https://redis.io/docs/latest/develop/data-types/hashes

• https://redis.io/docs/latest/commands/?group=hash

Sorted Sets (Zsets)

Zsets store unique strings with an associated score for ordering.

Example

> ZADD leaderboard 100 "Alice"
> ZADD leaderboard 200 "Bob"
> ZADD leaderboard 150 "Charlie"
> ZRANGE leaderboard 0 -1 WITHSCORES
1) "Alice"
2) "100"
3) "Charlie"
4) "150"
5) "Bob"
6) "200"

Usage

• Leaderboards

• Rate limiters

Important Notes

• Most sorted set operations are O(log(n))

• ZRANGE command's time complexity is O(log(n) + m), where m is the number of results returned.

• You can think of sorted sets as a mix between a Set and a Hash. Like sets, sorted sets are composed of unique, non-repeating string elements, so in some sense a sorted set is a set as well.

Helpful Links

• https://redis.io/docs/latest/develop/data-types/sorted-sets

• https://redis.io/docs/latest/commands/?group=sorted-set

Streams

Streams are an append-only log-like data structure.

Example

> XADD mystream * sensor-id 1 temperature 22.5
> XREAD COUNT 1 STREAMS mystream 0
1) "mystream"
2) 1) 1) "timestamp"
      2) "sensor-id"
      3) "1"
      4) "temperature"
      5) "22.5"

Usage

• Event sourcing (e.g., tracking user actions, clicks, etc.)

• Sensor monitoring (e.g., readings from devices in the field)

• Notifications (e.g., storing a record of each user's notifications in a separate stream)

Important Notes

• Adding an entry to a stream is O(1)

• Accessing any single entry is O(n), where n is the length of the ID.

• Consumer groups allow fan-out processing and acknowledgments

• Fan-out with Consumer Groups: Multiple consumers in a group distribute messages (load balancing), while independent consumers (outside a group) receive all messages.

Helpful Links

• https://redis.io/docs/latest/develop/data-types/streams

• https://redis.io/docs/latest/commands/?group=stream

Geospatial indexes

Store and query geographic locations.

Example

> GEOADD locations 2.3522 48.8566 "Paris"
> GEOADD locations -74.0060 40.7128 "New York"
> GEODIST locations Paris "New York" km
"5837.2690"

Usage

• Location-based services

• Nearby search

Important Notes

• O(log n) complexity for GEOADD, GEODIST, GEORADIUS

Helpful Links

• https://redis.io/docs/latest/develop/data-types/geospatial

• https://redis.io/docs/latest/commands/?group=geo

Bitmaps

Bit-level operations for efficient storage and tracking.

Example

> SETBIT visits:20240403 0 1
> SETBIT visits:20240403 1 1
> GETBIT visits:20240403 0
1

Usage

• Efficient set representations for cases where the members of a set correspond to the integers 0-N.

• Object permissions, where each bit represents a particular permission, similar to the way that file systems store permissions.

Important Notes

• SETBIT and GETBIT are O(1)

• BITOP is O(n)

Helpful Links

• https://redis.io/docs/latest/develop/data-types/bitmaps

• https://redis.io/docs/latest/commands/?group=bitmap

JSON

Redis JSON provides structured, hierarchical arrays and key-value objects that match the popular JSON text file format.

Example

> JSON.SET bike:1 $ '{"brand":"Deimos","model":"X1","colors":["red","black"]}'
OK
> JSON.GET bike:1
"{\"brand\":\"Deimos\",\"model\":\"X1\",\"colors\":[\"red\",\"black\"]}"
> JSON.GET bike:1 $.brand
"[\"Deimos\"]"

Usage

• Storing complex structured data (objects, arrays)

• Querying specific fields or nested values with JSONPath ($.field)

• Caching structured API responses

• Managing hierarchical data like configurations, product catalogs, or user profiles

Important Notes

• JSON operations are generally efficient, but performance depends on document size and depth

• JSON supports numbers, strings, booleans, nulls, arrays, and objects (full JSON spec)

• Memory usage can grow quickly if storing large documents (Danger)

Helpful Links

• https://redis.io/docs/latest/develop/data-types/json

Standalone

The simplest and default mode.

A single Redis instance handles all reads and writes.

Best for development, testing, or low-traffic production environments.

Master-Slave (Replication)

Master-Slave mode consists of N Redis instances, which can be one master with N slaves (if N masters with N slaves, it is not strictly a master-slave mode, and we will discuss it in the cluster mode. N+N Redis instances are required for N masters with N slaves). One function of the master-slave mode is to back up data so that data can be easily recovered when a node is damaged (meaning irreparable hardware damage) because there is a backup. Another function is load balancing. If all clients access one node, it will affect the efficiency of Redis's work. With master-slave, query operations can be completed by querying the slave node. Since master-slave replication means that the data of the master and slave are the same, there is a problem of data redundancy.

Sentinel

In a master-slave mode, when the master node goes down, the slave node can take over as the master node and continue providing service. However, there is an issue when the IP address of the master node changes. The application service still uses the original master node address to access, requiring manual intervention to modify. Sentinel can precisely solve this problem. Access to data in the Redis cluster is through the Sentinel cluster, which monitors the entire Redis cluster. Once a problem is detected in the Redis cluster, such as the master node going down, the slave node takes over. But when the master node address changes, the application service is unaware and does not need to change the access address because Sentinel interacts with the application service. Sentinel solves the failover problem well, taking high availability to another level. Of course, Sentinel has other functions, such as master node live detection, master-slave operation detection, and master-slave switching. A practical minimum configuration for Redis Sentinel is one master and one slave, monitored by at least three Sentinel nodes to ensure quorum and avoid split-brain.

Cluster

While master-slave replication provides data redundancy and Sentinel ensures automatic failover, these setups still have limitations. A single Redis node’s storage and access capacity is restricted, and scaling write operations across multiple nodes is not possible. Redis Cluster addresses these challenges by offering high availability, horizontal scalability, distribution, and fault tolerance.

For production-ready deployments, a Redis Cluster requires at least three master nodes, each of which can optionally have one or more replica nodes. This ensures data redundancy and allows the cluster to tolerate node failures without downtime. Cluster mode also enables automatic sharding, where data is divided across nodes, allowing the system to handle larger datasets and higher request throughput than single-node setups.

How does it work?

Data sharing is achieved through data sharding, while providing data replication and fault transfer functions. In the previous two modes, data was all on one node, and the storage capacity of a single node is limited. Cluster mode shards data and stores it on multiple nodes. When a shard reaches its limit, it is divided into various shards. Data is divided into 16384 slots (hash slots) in the key space of the cluster, and data is distributed to different shards based on hash, as follows:

HASH_SLOT = CRC16(key) % 16384

Data reading and writing after sharding: read requests are assigned to slave nodes, and write requests are assigned to the master node. Data is synchronized from the master to the slave node. Separating read and write requests improves concurrency and increases performance.

Horizontal expansion after data sharding: The master node can be expanded, and data migration is automatically completed within Redis. When you add a new master node, data migration is required, but the Redis service does not need to be offline.

For example, there are three master nodes, which means that Redis slots are divided into three segments, assuming the three segments are 0-7000, 7001-12000, and 12001-16383, respectively. Due to business needs, a new master node is added, and the four nodes collectively occupy 16384 slots. The slots need to be re-assigned, and the data must be migrated, but the service does not need to be offline. The Redis-trib management software within Redis performs the Redis cluster's re-sharding. Redis provides all the commands for re-sharding, and Redis-trib performs re-sharding by sending commands to the nodes.

Why does the Redis cluster have 16384 slots?

If the slot is 65536, the message header for sending heartbeat messages is too large, reaching 8 KB. As mentioned above, the largest space in the message header is myslots[CLUSTER_SLOTS/8]. When the slot is 65536, the size of this block is 8KB: 65536÷8÷1024=8KB. Since Redis nodes need to send a certain number of ping messages as heartbeat messages every second, if the slot is 65536, the message header of this ping message is too large and wastes bandwidth.

The number of Redis cluster master nodes cannot exceed 1000. As mentioned above, the more cluster nodes there are, the more data is carried in the message body of the heartbeat package. If there are more than 1000 nodes, it will also cause network congestion. Therefore, having more than 1000 Redis cluster nodes is not recommended. For Redis clusters with less than 1000 nodes, 16384 slots are enough. There is no need to expand to 65536. The smaller the slot, the higher the compression ratio with fewer nodes. In the Redis master node's configuration information, the hash slot it is responsible for is saved as a bitmap. During transmission, the bitmap is compressed. However, if the filling rate of the bitmap slots/N is high (N represents the number of nodes), the compression rate of the bitmap is low. If there are few nodes and many hash slots, the compression rate of the bitmap is low.

Example docker

name: redis-cluster
services:
  redis-cluster-0:
    container_name: redis-cluster-0
    image: bitnami/redis-cluster:latest
    ports:
      - 6379:6379
    environment:
      - 'ALLOW_EMPTY_PASSWORD=yes'
      - 'REDIS_NODES=redis-cluster-0:6379 redis-cluster-1:6380 redis-cluster-2:6381 redis-cluster-3:6382 redis-cluster-4:6383 redis-cluster-5:6384'
      - 'REDIS_PORT_NUMBER=6379'
      - 'REDIS_CLUSTER_ANNOUNCE_HOSTNAME=127.0.0.1'
      - 'REDIS_CLUSTER_ANNOUNCE_PORT=6379'
      - 'REDIS_CLUSTER_PREFERRED_ENDPOINT_TYPE=ip'
    networks:
      - redis-network

  redis-cluster-1:
    container_name: redis-cluster-1
    image: bitnami/redis-cluster:latest
    ports:
      - 6380:6380
    environment:
      - 'ALLOW_EMPTY_PASSWORD=yes'
      - 'REDIS_NODES=redis-cluster-0:6379 redis-cluster-1:6380 redis-cluster-2:6381 redis-cluster-3:6382 redis-cluster-4:6383 redis-cluster-5:6384'
      - 'REDIS_PORT_NUMBER=6380'
      - 'REDIS_CLUSTER_ANNOUNCE_HOSTNAME=127.0.0.1'
      - 'REDIS_CLUSTER_ANNOUNCE_PORT=6380'
      - 'REDIS_CLUSTER_PREFERRED_ENDPOINT_TYPE=ip'
    networks:
      - redis-network

  redis-cluster-2:
    container_name: redis-cluster-2
    image: bitnami/redis-cluster:latest
    ports:
      - 6381:6381
    environment:
      - 'ALLOW_EMPTY_PASSWORD=yes'
      - 'REDIS_NODES=redis-cluster-0:6379 redis-cluster-1:6380 redis-cluster-2:6381 redis-cluster-3:6382 redis-cluster-4:6383 redis-cluster-5:6384'
      - 'REDIS_PORT_NUMBER=6381'
      - 'REDIS_CLUSTER_ANNOUNCE_HOSTNAME=127.0.0.1'
      - 'REDIS_CLUSTER_ANNOUNCE_PORT=6381'
      - 'REDIS_CLUSTER_PREFERRED_ENDPOINT_TYPE=ip'
    networks:
      - redis-network

  redis-cluster-3:
    container_name: redis-cluster-3
    image: bitnami/redis-cluster:latest
    ports:
      - 6382:6382
    environment:
      - 'ALLOW_EMPTY_PASSWORD=yes'
      - 'REDIS_NODES=redis-cluster-0:6379 redis-cluster-1:6380 redis-cluster-2:6381 redis-cluster-3:6382 redis-cluster-4:6383 redis-cluster-5:6384'
      - 'REDIS_PORT_NUMBER=6382'
      - 'REDIS_CLUSTER_ANNOUNCE_HOSTNAME=127.0.0.1'
      - 'REDIS_CLUSTER_ANNOUNCE_PORT=6382'
      - 'REDIS_CLUSTER_PREFERRED_ENDPOINT_TYPE=ip'
    networks:
      - redis-network

  redis-cluster-4:
    container_name: redis-cluster-4
    image: bitnami/redis-cluster:latest
    ports:
      - 6383:6383
    environment:
      - 'ALLOW_EMPTY_PASSWORD=yes'
      - 'REDIS_NODES=redis-cluster-0:6379 redis-cluster-1:6380 redis-cluster-2:6381 redis-cluster-3:6382 redis-cluster-4:6383 redis-cluster-5:6384'
      - 'REDIS_PORT_NUMBER=6383'
      - 'REDIS_CLUSTER_ANNOUNCE_HOSTNAME=127.0.0.1'
      - 'REDIS_CLUSTER_ANNOUNCE_PORT=6383'
      - 'REDIS_CLUSTER_PREFERRED_ENDPOINT_TYPE=ip'
    networks:
      - redis-network

  redis-cluster-5:
    container_name: redis-cluster-5
    image: bitnami/redis-cluster:latest
    ports:
      - 6384:6384
    depends_on:
      - redis-cluster-0
      - redis-cluster-1
      - redis-cluster-2
      - redis-cluster-3
      - redis-cluster-4
    environment:
      - 'ALLOW_EMPTY_PASSWORD=yes'
      - 'REDIS_NODES=redis-cluster-0:6379 redis-cluster-1:6380 redis-cluster-2:6381 redis-cluster-3:6382 redis-cluster-4:6383 redis-cluster-5:6384'
      - 'REDIS_PORT_NUMBER=6384'
      - 'REDIS_CLUSTER_ANNOUNCE_HOSTNAME=127.0.0.1'
      - 'REDIS_CLUSTER_ANNOUNCE_PORT=6384'
      - 'REDIS_CLUSTER_PREFERRED_ENDPOINT_TYPE=ip'
      - 'REDIS_CLUSTER_REPLICAS=1'
      - 'REDIS_CLUSTER_CREATOR=yes'
    networks:
      - redis-network

  redis-insight:
    container_name: redis-insight
    image: redis/redisinsight:latest
    ports:
      - 5540:5540
    networks:
      - redis-network

networks:
  redis-network:
    driver: bridge

Helpful Definitions

Telegram’s API is blocked in Iran, making it inaccessible on all local hosting providers. This means that if your bot is hosted on an Iranian server, it cannot directly communicate with api.telegram.org.

I recently faced this issue while developing a Go-based Telegram bot. At first, in a simple solution, I tried using proxies inside my bot’s HTTP client, but I faced many issues and decided to use a better approach.

The Solution: Cloudflare Workers

Cloudflare Workers is a serverless platform that allows developers to run lightweight functions at the network edge. I decided to set up a Cloudflare Worker to act as a reverse proxy.

What is a Reverse Proxy?

A reverse proxy is a server that forwards requests on behalf of another server. In this case, instead of your bot directly calling:

https://api.telegram.org/bot<TOKEN>/getMe

It calls:

https://YOUR_WORKER_ADDRESS/bot<TOKEN>/getMe

The Cloudflare Worker then forwards the request to Telegram’s API and returns the response back to your bot. This completely bypasses any regional restrictions.

How to Set Up Cloudflare Workers for Telegram API

Step 1: Create a Cloudflare Worker

1. Go to Cloudflare Workers and create a new Worker.

2. Replace the default hello world script with the following code:

export default {
  async fetch(request) {
      const url = new URL(request.url);
      url.hostname = "api.telegram.org";
      
      const modifiedRequest = new Request(url, {
          method: request.method,
          headers: request.headers,
          body: request.method === "GET" ? null : request.body
      });

      return fetch(modifiedRequest);
  }
};

1. Save and deploy the Worker.

Step 2: Use the Created Worker

You just need to replace https://api.telegram.org with https://YOUR_WORKER_ADDRESS in any framework or language. For example, in Go:

var (
    Bot              *tgbotapi.BotAPI
    TelegramBotToken string
)

func InitTelegram() {
    TelegramBotToken = os.Getenv("TELEGRAM_BOT_TOKEN")
    proxyURL := os.Getenv("TELEGRAM_PROXY_URL") // https://YOUR_WORKER_ADDRESS
    var err error
    Bot, err = tgbotapi.NewBotAPIWithAPIEndpoint(TelegramBotToken, proxyURL+"/bot%s/%s")
    if err != nil {
        log.Fatal(err)
        return
    }
    log.Printf("Authorized on account %s", Bot.Self.UserName)
}

Why Use Cloudflare Workers?

• No need for a VPS

• Fast and free (With some limitation)

• Easy to deploy and maintain

• Bypasses regional restrictions effortlessly

Conclusion

If you are facing Telegram API restrictions in Iran or any other country, Cloudflare Workers is a powerful and free solution to keep your bot running smoothly. By simply rerouting requests through a Worker, you can avoid direct blocks and enjoy uninterrupted access to Telegram’s API.

Have you tried this solution? Let me know your experience in the comments!

What is Map Tools & Who is it for?

Map Tools is an online map assistant that provides utilities such as geometry extraction, coordinate conversion, and WKT (Well-Known Text) visualization. Whether you're a GIS professional, developer, or anyone working with spatial data, this tool makes these tasks easier and more efficient.

I built Map Tools to streamline repetitive tasks I often encountered in GIS and location-based projects.

Check it out

GitHub: github.com/abmohajeri/map-tools

Live Demo: abmohajeri.github.io/map-tools

Contributions & Support

If you find Map Tools useful, give it a ⭐ on GitHub! Your support helps improve the project.

I’d also love your feedback—try it out, fork the repo, submit issues, or create pull requests.

Let’s build something great together! 🚀

With the rise of photogrammetric tools and software, creating highly detailed 3D models has become easier and more accessible. Photogrammetry combines photography, geometry, and advanced algorithms to deliver models that are accurate and visually stunning.

In this article, we’ll dive into the basics of photogrammetry, how it works, and some simple examples that showcase its potential. We’ll also take a look at demo of my own photogrammetry model, Fotros, available on Hugging Face here, which demonstrates the process in action.

How Photogrammetry Works

At its core, photogrammetry relies on multiple photos taken from different angles of the same object or scene. By using triangulation, the software can calculate the distance and positions of points on the object from these images. Here’s a simplified breakdown of the process:

1. Image Acquisition: Multiple photos are taken from different viewpoints. Ideally, the object or scene should be captured from various angles to ensure the maximum amount of detail is recorded.

2. Processing: The photos are fed into photogrammetry software that analyzes the overlapping areas in the images. Key points (or "features") in the images are identified and matched across multiple photos.

3. Triangulation: Using the positions of the cameras and the identified matching features in the photos, the software calculates the relative positions of these points in 3D space, effectively reconstructing the object.

4. 3D Model Creation: Finally, the software generates a point cloud—a collection of 3D coordinates representing the object. These points can be further refined into a 3D mesh and textured to create a realistic 3D model.

Here is example of multiple photos taken from different angles of the same object:

List of the best photogrammetry apps

Here’s a list of the best photogrammetry apps:

1. RealityCapture: A fast and efficient photogrammetry tool, popular for handling large datasets and high-resolution photos. It is used for creating highly accurate 3D models of landscapes and objects.

2. Agisoft Metashape: A professional-grade software known for its high accuracy and flexibility, supporting small and large projects for industries like archaeology, architecture, and aerial mapping.

3. 3DF Zephyr: A versatile photogrammetry tool that offers both free and paid versions. It is used to create precise 3D reconstructions of objects, buildings, and landscapes.

4. Meshroom: An open-source photogrammetry tool that allows users to create detailed 3D models from photos. It is user-friendly and suitable for both beginners and professionals.

Real-World Application: Fotros

To showcase the power of photogrammetry, I developed a model called Fotros, which is available on Hugging Face. Fotros allows users to upload photos of an object from various angles, and it generates a 3D model using photogrammetry techniques.

This model is a practical example of how easy it can be to turn photos into 3D models. It uses Meshroom as the backend for photogrammetry and is integrated with advanced algorithms to make the process fast and accurate. It’s ideal for beginners and professionals alike who are interested in creating 3D models for augmented reality (AR), virtual reality (VR), and other visualizations.

Conclusion

Photogrammetry is an incredible tool that turns ordinary photos into precise 3D models, opening the door for countless applications. Whether you're documenting an archaeological site or creating 3D assets for video games, photogrammetry offers a powerful solution.

With models like Fotros, the process is becoming more accessible than ever. If you're interested in seeing photogrammetry in action, I encourage you to try out Fotros and start exploring the world of 3D modeling!

Overview of BERT

BERT (Bidirectional Encoder Representations from Transformers) is a groundbreaking model developed by Google for natural language understanding tasks. It is designed to understand the context of words in a sentence by looking at the words before and after them. BERT has revolutionized the field of NLP because it is pre-trained on a vast amount of text and can be fine-tuned for specific tasks like question answering, sentiment analysis, and more.

Overview of Question Answering Systems

Question Answering (QA) systems are a type of AI that can automatically answer questions posed by humans in natural language. These systems typically involve two main components:

1. Contextual Understanding: The model comprehends the context of the input text (e.g., a paragraph).

2. Answer Extraction: The model identifies and extracts the relevant answer from the context based on the given question.

In the context of Persian language processing, building such systems poses unique challenges due to the complexity of the language, as well as the limited availability of high-quality annotated data.

Explainable Persian QA with BERT

In this project, i implemented a Persian QA system using BERT, specifically tailored for the Persian language. What sets this work apart is the focus on explainability—ensuring that the model’s decisions and answer predictions are transparent and understandable to users. This is particularly important in sensitive applications where understanding the reasoning behind an AI’s response is crucial.

Access the Code

The code for this model, along with detailed documentation, is available on GitHub. he repository includes all the necessary resources for implementing the explainable Persian QA system.

GitHub Repository

Presentation at ICCKE 2022

This project was presented at the International Conference on Computer and Knowledge Engineering (ICCKE) in 2022. The conference provided an excellent platform to share and discuss cutting-edge research with peers from around the world.

If you are interested in a more in-depth explanation, including the technical aspects of the model, you can watch the video of my presentation, conducted in Persian:

https://www.youtube.com/embed/9mQxD6wWD80

Overview of the CNN Model

CNNs have become the go-to models for image classification tasks due to their ability to automatically and adaptively learn spatial hierarchies of features through backpropagation. The model I developed is designed not only for accuracy but also for interpretability—providing insights into why the model makes certain predictions.

I used Fashion MNIST dataset. The Fashion MNIST dataset is a popular alternative to the traditional MNIST dataset, featuring 70,000 grayscale images of 10 different categories of clothing items, such as shirts, sneakers, and dresses. The challenge lies in correctly classifying these items into their respective categories.

Explainability

Explainability in AI is about making the inner workings of machine learning models transparent. This is especially important in fields where trust and accountability are paramount. By incorporating explainability techniques, the model provides a clearer understanding of how it reaches its conclusions.

I explain the concept of Explainable AI in this article.

Access the Code

The code for this model, along with detailed documentation, is available on GitHub. It includes not only the implementation of the CNN but also the methods for visualizing and explaining the results.

GitHub Repository

Presentation at ICCKE 2022

This project was presented at the International Conference on Computer and Knowledge Engineering (ICCKE) in 2022. The conference provided an excellent platform to share and discuss cutting-edge research with peers from around the world.

If you are interested in a more in-depth explanation, including the technical aspects of the model, you can watch the video of my presentation, conducted in Persian:

https://www.youtube.com/embed/G01Xtzp_tzM

Explainable Artificial Intelligence

In the past few years, a hot and important concept called artificial intelligence has been expressed, which generally refers to the methods and techniques that can be used to explain how artificial intelligence works and how to clarify the decision-making of black box models for humans. In general, these explanations provide an answer to the question of why a model predicts an outcome. For example, consider the following model:

As shown in the figure above, the process of many machine learning models is that a black box is first trained by the dataset and that black box is used to make predictions on new inputs. It is clear that the function of the black box is not obvious, therefore, explainable artificial intelligence adds clarity to such models by providing a variety of explanations and turning them into a transparent model like the one below:

Explanations and their types

As mentioned earlier, providing explanations is a way to add transparency to various AI black box models. Computer scientists have tried for decades to open these black boxes and increase the transparency of such models by providing explanations. A recent research review shows that these efforts have yielded good results. Therefore, explanations can be considered as information about the results and model predictions, which helps to better understand the performance of the model. These explanations can be presented in the following types:

• Textual explanations: In this type of explanation, a textual explanation is provided that shows the function of the model. Texts in this type of explanation can be simply presented or generated by the learning process. For example, the output of \(x_1\) is equal to \(y_1\) because \(x^3>20\).

• Model simplification: In this type of explanation, a completely new model is reconstructed based on the previously trained model. This new and simplified model tries to optimize the previous complex model and reduce its complexity by maintaining its performance. For example, part a of the figure below shows this type of explanation.

• Visualization: Visualization means being able to display and visualize the behavior of the model and its complex interactions through a visual display, for example, in the form of charts. For example, part b of the figure below shows an example of this explanation.

• Local explanations: This explains the part performance of the entire system by dividing the model space into smaller subspaces and examining them.

• Feature importance: This explanation calculates the importance of a feature on the model prediction, comparing and checking the importance of this feature in different models can give very good information about the performance of those models. Part c of the figure below refers to this type of explanation.

• Explanation by example: This means that the sample extracted data from the model by which we can have a better understanding of the model performance. For example, input 3 leads to output 10.

Conclusion

By utilizing these explanation techniques, Explainable AI (XAI) not only enhances the transparency of AI models but also builds trust in AI-driven systems, making them more reliable for critical decision-making processes. As AI continues to integrate into various aspects of our lives, the importance of explainability cannot be overstated. It ensures that AI is not just powerful but also understandable and accountable.

For practical examples of XAI in action, you might be interested in exploring the following articles:

• Explainable CNN Model on the Fashion MNIST Dataset

• Explainable Question Answering System

These articles delve into specific applications of XAI, offering insights into how explainability is applied in real-world scenarios.