Database drivers, ADO.NET and Redis - talking to databases and caches safely
How .NET talks to a database through ADO.NET - pooled connections, commands, parameters, readers and transactions - why parameters stop SQL injection, how one driver contract covers PostgreSQL, SQL Server and SQLite, how Redis stores, expires and evicts data, why SCAN beats KEYS, what a multiplexer is, and how to run a query you did not write without letting it change anything.
In short
Every program that keeps data talks to its database through a driver - a library that speaks that database's network protocol. In .NET, the drivers for PostgreSQL, SQL Server and SQLite share one shape called ADO.NET: a connection borrowed from a pool, a command with its parameters, a data reader and a transaction. Redis is a different kind of store: an in-memory server of typed keys that can expire, reached from .NET through StackExchange.Redis. {{brand}} uses all of them, and its SQL Tester and Cache Tester point them at a team's own database or cache - so they show how to do it with care.
After this lesson you can Write correct ADO.NET code with pooled connections, parameters, readers and transactions. Explain SQL injection, and why a rollback is a safety net and not a wall. Design one driver contract for several databases. Use Redis well: the right data type, a TTL on every cached key, SCAN instead of KEYS, one shared multiplexer and a hit rate you can read.
Words you need
- Driver (data provider) A library that speaks one database's wire protocol: Npgsql for PostgreSQL, Microsoft.Data.SqlClient for SQL Server, Microsoft.Data.Sqlite for SQLite.
- ADO.NET .NET's common data-access classes - DbConnection, DbCommand, DbParameter, DbDataReader, DbTransaction - that every driver implements. EF Core and Dapper are built on top of it.
- Connection pool Open connections a driver keeps and lends out. Open borrows one, Dispose gives it back; there is one pool per exact connection string.
- Parameter A value sent beside the SQL text, with its own type. The server never reads it as SQL.
- SQL injection An attack where input is pasted into the SQL text and changes what the statement does.
- Data reader A forward-only stream of rows, read one at a time; one command can return several result sets.
- Transaction Statements that all commit or all roll back together; other sessions do not see the changes until commit.
- Adapter (driver pattern) One interface your code depends on, with one class per vendor that translates it to that vendor's library.
- Redis An in-memory key-value server whose values are typed: strings, hashes, lists, sets, sorted sets and streams.
- TTL Time to live: seconds until a key expires. Redis answers -1 for a key that never expires and -2 for a key that does not exist.
- Multiplexer StackExchange.Redis's one shared connection object; every caller's commands are pipelined over it.
- Hit rate Hits divided by hits plus misses: how often a cache lookup found what it asked for.
How ADO.NET talks to a database
Whatever the database, an ADO.NET conversation has the same seven steps. Learn them once and you can read code for any driver, and you will understand what EF Core and Dapper do for you underneath.
- 1. Open (borrow) OpenAsync takes a connection from the pool, or opens a new one: TCP, TLS and a login. Only the first open costs real time. (With Npgsql 7 and later you ask an NpgsqlDataSource, which owns the pool.)
- 2. Begin a transaction BeginTransactionAsync starts a unit of work. Without one, every statement commits on its own (autocommit).
- 3. Build the command CommandText holds SQL with placeholders such as @customer; CommandTimeout limits how long the server may work; the command joins the transaction.
- 4. Bind parameters Each value becomes a DbParameter with a name, a type and a direction: input, output, input-output or return value.
- 5. Execute ExecuteReaderAsync for rows, ExecuteNonQueryAsync for the number of rows affected, ExecuteScalarAsync for the first cell of the first row.
- 6. Read ReadAsync moves to the next row and NextResultAsync to the next result set; RecordsAffected says how many rows a write touched.
- 7. Finish and give back Commit or roll back, then dispose: the connection goes back to the pool, ready for the next caller.
Output values arrive last SQL Server sends a procedure's output parameters and its return value after all the rows. Read them once the reader is closed - inside the reading loop they are still empty. Code that reads every result set first and the outputs afterwards works on every driver.
Connection pooling, inside
Opening a real connection is slow: a TCP handshake, a TLS handshake and a login, often tens of milliseconds - far longer than a simple query. So every ADO.NET driver keeps a connection pool. Your code still writes open, use, dispose around every piece of work; the driver quietly turns that into borrow and return.
- First open No idle connection is waiting, so the driver connects for real and logs in.
- Dispose The connection is parked in the pool, still connected; its session state is reset before anyone uses it again.
- Next open The driver lends the parked connection at once - no network round trip.
- Pool full When every connection is in use (100 by default in Npgsql and SqlClient), Open waits, then fails with a timeout. This is pool exhaustion.
- Idle and broken Connections idle for too long are closed, and a broken one is thrown away. ClearPool empties a pool by hand, for example after a failover.
- Open late, close early: borrow just before the work and dispose right after, never across a wait for a person or another service.
- Always await using: a connection that is never disposed never returns, and after a hundred of them the next Open times out.
- One connection per thread: a DbConnection is not thread-safe; parallel work borrows one connection each from the same pool.
- One connection string per database: a stray option or a changed application name quietly makes a second pool.
- Npgsql 7 and later: build one NpgsqlDataSource per database at start-up and keep it; it owns the pool and the settings.
- Timeouts and cancellation are different: CommandTimeout makes the server give up after a number of seconds, while a CancellationToken lets your code abandon the call early. Set both, so a stuck query frees its connection.
Read a load test with the handshake in mind Any load test against a database pays for the connection handshake on its first request, because the pool is empty. Read the slowest time with that in mind. {{brand}}'s own data reaches PostgreSQL through EF Core on Npgsql's pool.
Transactions: all or nothing
A transaction groups statements so that they succeed or fail as one. Databases keep that promise with a write-ahead log and locks; you meet it as four letters, ACID.
If a transaction is disposed without CommitAsync, the driver rolls it back. Do not rely on that for clarity, but know it is the safety net.
- Retrying execution strategy: EF Core can retry a whole unit of work after a short outage, so it refuses a transaction you start by hand. Wrap the transaction and everything in it inside the execution strategy, so a retry runs it again from the start.
- Racing inserts: two requests can insert the same unique value at once. Let the unique index decide, catch the violation (PostgreSQL SQLSTATE 23505), and read the winner's row instead of failing.
- Dirty read You read another transaction's change that is later rolled back.
- Non-repeatable read You read the same row twice and get two answers, because someone committed in between.
- Phantom read You run the same query twice and a new row appears.
- Begin Open a transaction before the statement you want to try.
- Run and look Run it, read every result and check it: inside the transaction the change is visible to you.
- Roll back Undo everything. Other sessions never saw the change.
- Know the limits Some things happened anyway: numbers taken from sequences, work done outside the database, and locks held while it ran.
{{brand}}'s SQL Tester rolls back by default A test is a question, not a change, so {{brand}}'s SQL Tester runs statements inside a transaction that it rolls back by default. A rollback is a safety net, not a wall: teams should still give a tester a read-only login, the one wall no statement can climb.
SQL injection and parameters
SQL injection is still one of the most common security bugs - injection has a place in the OWASP Top 10. It happens when input becomes part of the SQL text. Parameters fix it at the root: the text and the values travel separately, so a value can never turn into SQL.
- The input A search box receives x' or '1'='1.
- Concatenation The code builds the query by pasting the input between quotes, so the input's own quote closes the string early.
- A new meaning The server now reads where name = 'x' or '1'='1' - a condition that is always true.
- The damage Every row comes back. With a UNION or a second statement, the attacker can read or destroy other data.
Why it is safe: PostgreSQL's extended protocol sends Parse (the text with numbered placeholders), then Bind (the values), then Execute - the parser has finished before any value arrives. SQL Server sends a parameterized command as a remote procedure call with typed values, the same thing sp_executesql does. A bonus: the server can reuse the plan of the same text with new values.
Parameters protect values, not statements A tool that runs SQL a person wrote cannot parameterize the statement - the statement is the input. That is why such tools need other layers: who may run what, where it may connect, what the text may contain, and a transaction around it. If your own design writes parameter names into the SQL text, validate them as plain identifiers first.
How Redis works
Redis keeps every key in memory and executes commands on one main thread, one after another. That makes each command atomic and very fast - usually well under a millisecond - and it also means one slow command makes every other client wait. Clients talk to it over TCP with a simple protocol called RESP.
- Written SET page:home ... EX 300 stores the value with a TTL of 300 seconds.
- Read A GET that finds the key is a hit; a GET for a key that is not there is a miss.
- Expired Redis removes expired keys lazily, when someone touches them, and actively, by sampling keys with a TTL in the background.
- Evicted When memory reaches maxmemory, the eviction policy picks keys to drop - or, by default, refuses new writes.
- noeviction (the default policy): nothing is dropped, and once memory reaches maxmemory writes fail with an error. By default maxmemory is 0, which means no limit at all, so set one for a cache.
- allkeys-lru and allkeys-lfu: drop the least recently or least frequently used key - the usual choice for a cache.
- volatile-lru, volatile-lfu and volatile-ttl: drop only keys that have a TTL, the last one choosing the key closest to expiring.
- allkeys-random and volatile-random: drop a random key, from all keys or from those with a TTL.
- Ask Redis GET the key. A hit answers at once.
- On a miss, ask the source Read the database: the slow but true copy.
- Fill with a TTL SET the value with an expiry, so a stale copy cannot live forever.
- Fail open If Redis is down, skip it and serve from the source: slower, never failed.
- Invalidate When the data changes, delete the key, or let its TTL end it.
When many requests miss at once If a popular key expires, hundreds of requests can miss together and all ask the database: a cache stampede. Spread expiry times with a little random jitter, let one request rebuild the value while the others wait, or refresh a hot key shortly before it expires. Also cache "not found" answers for a short time, so requests for ids that do not exist do not hit the database every time.
An application that runs several copies needs a shared cache such as Redis; with one copy, its own memory is enough. {{brand}} follows the same pattern for its own cache: when the cache cannot be reached, requests become slower, not failed.
The options in the market
- ADO.NET directly Connection, command, parameters and reader, by hand. Chosen for running SQL that people write.; Full control of transactions, timeouts and isolation; Every result set, output parameter, return value and server message; Nothing between you and the driver; More code: you map rows yourself; Easy to forget a dispose or a parameter
- Dapper A micro-ORM: extension methods on DbConnection that map rows to objects.; Very fast, very little code; Still raw SQL, still parameterized; Built for result shapes you know in advance; A tool that runs any SQL gains little from it
- EF Core A full ORM: LINQ, change tracking and migrations. Chosen for the product's own data.; Queries in C#, checked by the compiler; Migrations and global query filters; The database provider can be swapped; Hides the SQL it sends; Not made for running SQL someone else wrote
- ODBC or OLE DB Older generic bridges: one API for many databases.; Reaches almost any database, legacy ones included; Native drivers to install on every machine; Weaker async support and fewer modern features
- Redis The in-memory server most teams run for caches and sessions.; Rich data types and a TTL on every key; Persistence, replication and pub/sub; Offered as a managed service by every large cloud; Bounded by memory; One slow command blocks everyone; Its licence changed in 2024; Redis 8 added the AGPL as an option
- Valkey The Linux Foundation's fork of Redis 7.2, BSD licensed.; An open licence; The same protocol and commands, so clients work unchanged; A younger project; Features added to Redis after the fork may differ
- Memcached A simple, multi-threaded cache of strings.; Very fast for plain values; Uses every CPU core; Strings only, no data types; No persistence and no replication built in
- In-process memory IMemoryCache inside each server process.; The fastest possible lookup; Nothing to run or secure; Every server copy has its own, so copies disagree; Lost on every restart
What {{brand}} chose, and why
- ADO.NET by hand for other people's databases. A tester must run whatever SQL a person writes and keep everything that comes back: several result sets, output parameters, return values, server messages, rows affected and exact errors. A mapper adds nothing there, and raw ADO.NET gives full control of transactions and timeouts.
- EF Core for its own data. {{brand}}'s own tables use EF Core on PostgreSQL, with migrations and query filters that keep every organization's rows apart.
- One driver contract, many vendors. {{brand}} puts every database behind one interface, so adding a database means writing one new adapter.
- Redis because it is what teams actually run for caches and sessions, and one protocol covers managed services and self-hosted servers alike.
The libraries behind those choices:
The trade-off accepted More hand-written code and more care: every connection disposed, every vendor's errors mapped into one list, and a database is supported only once someone writes its driver. Choose Dapper for an application whose result shapes you know, EF Core for your own domain model, and Valkey if an open licence matters more than the newest Redis features.
One contract, many databases: the driver pattern
The adapter pattern puts one interface between your code and the vendors. The calling code asks to run a request; each adapter translates that into its own library. ADO.NET already makes half of the work vendor-neutral, so a good design writes the shared part once in a base class and lets each vendor fill in only what really differs.
- Write the adapter A new class on the base: create the connection, call a procedure the vendor's way and classify its errors.
- Decide what the connection string may say Decide which connection options your tool accepts. Options that point somewhere else (several hosts, failover servers, local sockets) can step around your address checks.
- Register it Add it to dependency injection beside the others; the calling code picks one by its provider name.
- Learn its dialect If you inspect statements, learn the new database's quoting and comment rules before you rely on the inspection.
- Test against a real server Run real SQL end to end against a throwaway server, for example one started in a container.
Error codes are a vendor's real API Map SQLSTATE codes (PostgreSQL: 23505 unique violation, 40P01 deadlock) and SQL Server error numbers (2627 unique key, 1205 deadlock) into one list of your own. Then expecting a constraint violation means the same thing on every database.
Reading a Redis server without hurting it
Because Redis runs one command at a time, the most dangerous command is a long one. KEYS pattern walks every key in the database before it answers - on a server with ten million keys that can freeze it for seconds. SCAN does the same walk in small pages, so other clients keep being served in between.
- Start at 0 SCAN 0 MATCH session:* COUNT 100 returns a next cursor and a page of keys.
- Follow the cursor Call SCAN again with the cursor it gave you; each call does a small, bounded piece of work.
- The filter comes late MATCH is applied after keys are fetched, so a page can be empty while the scan is not finished. COUNT is a hint, not a promise.
- Stop at 0 The scan is complete when the cursor comes back as 0. A key can appear twice, and keys added during the scan may or may not appear.
INFO returns the server's health as text sections - server, clients, memory, persistence, stats, replication, keyspace. A few numbers tell most of the story, and the hit rate is the one every cache owner should know.
- The counters run since the server started (or since CONFIG RESETSTAT), so compare two readings taken some minutes apart.
- A low rate usually means keys expire too soon, are never read twice, or carry a name that changes on every request.
- Rising evicted_keys means maxmemory is too small for the data that is actually used.
- No lookups yet means no rate at all - an empty cache is not a 0% cache.
- Connect once Create one ConnectionMultiplexer at start-up and register it as a singleton. Creating it is expensive; sharing it is the design.
- GetDatabase is cheap Call GetDatabase wherever you need it - it is a light wrapper, not a new connection.
- Commands are pipelined Calls from every thread go out over the same socket without waiting for each other, and replies are matched back in order.
- Batches and transactions CreateBatch sends several commands together, and each command still returns its own result; CreateTransaction wraps them in MULTI and EXEC, with optional conditions.
- Pub/sub on its own link Subscriptions use a second connection, because a subscribed connection cannot run other commands.
- It reconnects for you With abortConnect=false the multiplexer starts even while Redis is down and reconnects in the background.
- Never expose the port to the internet: keep Redis on a private network and reach it through a firewall rule or a private link.
- Require authentication: set a password (requirepass) or, better, ACL users that may run only the commands they need.
- Turn TLS on when traffic crosses a network you do not control, and keep certificate checks on.
Where {{brand}} uses it Cache Tester browses keys with SCAN and fetches the type and TTL of a whole page of keys in one batch, so looking at a busy cache does not slow it down.
Running a query you did not write
A SQL testing tool, an admin console and an AI assistant that writes queries all face the same problem: the statement is untrusted input. No single check is enough, so safe systems stack layers - each one cheap, each one catching what the others miss.
- Loopback The machine itself (127.0.0.1); a tool that connects there reaches its own server.
- Private address An internal network address such as 10.x.x.x or 192.168.x.x, not reachable from the internet.
- Cloud metadata address A special internal address (169.254.169.254) where a cloud server can ask for its own credentials.
- DNS rebinding A name that answers with a safe address when you check it and an internal one when you connect.
- Who Check that the person may run on this environment at all.
- Where Allow only expected hosts. Resolve the name once, refuse internal addresses, and connect to the address you checked, so a DNS answer that changes in between cannot redirect you. Never let the tool reach your own database.
- What Inspect the text with comments and string literals removed, and refuse what your tool has no business running.
- How Run inside a transaction that is rolled back, with a statement timeout and limits on the rows and characters kept.
- After Judge the real result, then mask secrets before anything is stored or shown, and record who ran what.
- Underneath Give the tool a read-only database login. It is the one layer no clever text can talk its way past.
- Check The name resolves to a public address and passes the check.
- Switch The attacker's DNS now answers with an internal address.
- Connect A driver that resolves the name again connects inside the network. The fix: connect to the address you checked.
Text filters are never complete, which is why the last layer is a read-only login.
- Roll back a transaction Run, read, undo.; The database undoes it, even after a crash; The test sees its own changes, so checks work; Sequences, outside effects and locks still happen
- Commit, then clean up Run for real, then a script removes what was added.; Can test behaviour that needs a commit; A failed clean-up leaves the data behind; Other sessions saw the change
- Restore a snapshot Copy the database before, restore it after.; Undoes everything, structure included; Slow for large databases; Wipes what others did meanwhile
- A throwaway container A fresh database per test run.; Perfect isolation; Not the real data, so not the real question
- A read-only login The database refuses every write.; No text can get around it; Needs the database owner; Cannot test procedures that write
When an AI assistant writes the query, use an allow-list rather than a deny-list: accept one statement that starts with SELECT or WITH and contains nothing that writes, locks, waits or reaches outside the database, and still run it in a rolled-back transaction.
- Give it no access to the host's disk or to other files.
- Do not let it load code.
- Put hard limits on time, size and memory, so one bad query fails alone and never takes the host down.
- Test the fences with hostile statements, not only friendly ones.
Try it yourself: a drivers lab in an hour
All you need is Docker and the .NET SDK. You will see a rollback undo an insert, an injection return every row while a parameter returns none, and Redis report its hit rate.
- Start the servers Run PostgreSQL and Redis in containers, wait a few seconds for them to start, and create a small orders table.
- Create the project A console app with the Npgsql and StackExchange.Redis packages.
- Watch a rollback Insert inside a transaction, count the rows inside it, roll back and count again.
- Try the injection Paste x' or '1'='1 into the text, then send the same input as a parameter.
- Fill Redis and scan it Create keys with redis-benchmark, walk them with SCAN and read INFO stats.
Go one step further Run docker exec -it redis redis-cli --latency in a second terminal, then docker exec -it redis redis-cli keys '*' in a third, and watch the latency jump while KEYS walks every key. With 180,000 keys expect a spike of tens of milliseconds; with millions of keys it is seconds. Then run the program's SCAN loop and watch the latency stay flat. You have just seen why KEYS is banned on production servers.
Common mistakes
- Building SQL with + or string interpolation, even from values you think are safe, such as a column read from another query. Use parameters for values, and an allow-list or proper quoting for names.
- Not disposing connections and readers. The pool runs dry and every request starts to time out. Write await using every time, and open late, close early.
- Creating a ConnectionMultiplexer per request. Each one is a slow handshake and new sockets; under load the server drowns. Create one and share it.
- Running KEYS, or HGETALL on a huge key. Redis stops serving everyone while it works. Use SCAN, HSCAN and paged reads.
- Treating a rollback as complete protection. Sequences, outside effects and locks still happen. Give testing tools a read-only login as well.
- Caching with no TTL and no outage plan. Stale data lives forever, and a Redis failure becomes a site failure. Put a TTL on every cache key, fall back to the source, and give hot keys a randomised TTL so they do not all expire together.
Interview questions
- What is ADO.NET, and how do EF Core and Dapper relate to it? ADO.NET is .NET's base data-access layer: connections, commands, parameters, readers and transactions that every driver implements. Dapper adds object mapping on top of a DbConnection; EF Core adds LINQ, change tracking and migrations, and still talks through an ADO.NET driver underneath.
- How does connection pooling work, and how can it break? Open borrows a live connection from a pool keyed by the exact connection string, and Dispose returns it, so only the first open pays for TCP, TLS and login. It breaks when code forgets to dispose - the pool runs dry and Open times out - or when slightly different connection strings create many small pools.
- Why do parameters prevent SQL injection? Can everything be a parameter? The statement is parsed before the values arrive, so a value can never change what the statement does. Only values can be parameters; table and column names or a sort direction need an allow-list or the database's identifier quoting.
- What are the isolation levels, what is the default in PostgreSQL and SQL Server, and what is a deadlock? Read committed is the default in both: you never read uncommitted data, but two reads may differ; repeatable read and serializable add stability, and you retry serialization failures. A deadlock is two transactions each holding a lock the other needs, so the database aborts one (SQLSTATE 40P01, SQL Server error 1205); the work is rolled back, so retry the whole transaction after a short random delay.
- Can a rollback undo everything a test did? No. Sequence numbers stay used, effects outside the database stay done, and locks and load already happened. Treat rollback as a safety net and give testing tools a read-only login.
- What is a cache stampede and how do you prevent it? When a popular key expires, many requests miss together and all hit the database at once. Add random jitter to expiry times, let one request rebuild the value while the others wait, or refresh a hot key shortly before it expires.
- Why is KEYS dangerous, and how does SCAN avoid the problem? Redis executes commands one at a time, and KEYS walks the whole keyspace in one go, blocking every other client. SCAN walks it in small pages driven by a cursor, so each call is short; you stop when the cursor comes back as 0.
- Why should a ConnectionMultiplexer be shared? Creating one is expensive, and it is thread-safe by design: commands from every caller are pipelined over one socket and replies are matched in order. One per request means slow handshakes and socket storms; one per application is the intended use.
- How do you compute a cache hit rate, and what does a low one tell you? keyspace_hits divided by hits plus misses from INFO stats, compared between two readings because the counters run since start-up. A low rate means keys expire too soon, are never reused or have names that change on every request; rising evicted_keys means memory is too small.
- How do you run a query you did not write without letting it change anything? Stack layers: check who may run it and where it may connect, inspect the text with comments and literals removed, run it in a transaction that is rolled back with a timeout and row limits, and mask what you keep. Underneath it all, use a read-only database login.
Related
- /help/technology/data
- /help/technology/http-api-testing
- /help/technology/realtime-events