Another on-the-fly post about some code opinions I have. Inspiration struck, and since the last time I felt this and acted on it was over 6 months ago, I decided to ride the wave!
I’ve been thinking about the adapter pattern lately. Some may know I am a fan of functional programming, particularly beause of how well it lends itself to unit testing and test-driven development.
A coworker introduced me to the adapter pattern a couple of years ago, and I love it. It’s essentially an elegant way to inject methods as inputs into a larger function, and is particularly helpful for a few reasons: 1 - implementing the same basic functionality for different domains 2 - testing with very lightweight mocks. This is particularly helpful when you’re working with serverless functions, where you may want to test all the barebones of functionality locally.
Today, I discovered a third one: managing memory/API calls.
First, let me explain the context.
# 1. What are adapters?
Here’s my definition without looking it up: I think of adapters as a functional programming alternative to the convenience of classes in object-oriented development. It allows to create one repeatable, reusable set of functionalities, but without state management and the complexities it introduces.
We work a bunch with external APIs, and particularly data warehouses, so I’ll stick with that domain because I’m not capable of thinking of something else. Here’s an example, in Typescript, of such an adapter pattern.
OK, so say we have a function that should check on a table in a data warehouse, see whether it exists and get its row count, and then write back that info somewhere.
We’ll first write our adapters type, as well as our factory function, which will take in the adapters and spit out a function that implements those adapters.
import { Logger } from 'winston'; // great logger for Node.js
type TableState = {
exists: true;
rowCount: number;
} | {
exists: false;
}
type Adapters = {
countRows: (tableName: string) => Promise<number>;
checkIfTableExists: (tableName: string) => Promise<boolean>;
writeTableState: (tableName: string, state:TableState) => Promise<void>;
getLogger: () => Logger;
}
function handlerFactory(adapters: Adapters): (tableName: string) => Promise<void> {
const logger = adapters.getLogger();
async function reportTableState(tableName: string) {
const tableExists = await adapters.checkIfTableExists(tableName);
if (!tableExists) {
logger.info(`Table does not exist.`);
return adapters.writeTableState(tableName, { exists: false });
}
const rowCount = await adapters.countRows(tableName);
logger.info(`Table exists with ${rowCount} rows.`);
return adapters.writeTableState(tableName, { exists: true, rowCount });
}
return reportTableState;
}
# 2. How do you use them across different domains?
Let’s implement those adapters for e.g. Databricks, and hopefully it’s clear enough how it can be reused for other data warehouses like Snowflake or BigQuery.
import { upsertTableState } from './internal-firebase-impl'; // say you have a method that takes in firebaseDb as a first arg
import { firebaseDb } from 'firebase';
import { getLogger } from 'utils';
import { getDatabricksClient } from 'databricks-pkg';
const databricksClient = getDatabricksClient({auth});// something that implements Databricks API, logs in, etc
const databricksAdapters: Adapters = {
countRows: (tableName:string) => databricksClient.countRowsInTable(tableName), // illustrating a case where the databricksClient is a class with methods
checkIfTableExists:(tableName:string) => databricksClient.doesTableOrViewExist(tableName, type: 'table'), // illustrating that you can pass other parameters in the adapter implementation
writeTableState: (tableName: string, state:TableState) => upsertTableState(firebaseDb, {tableName, state}),
logger: () => getLogger({context: {dataWarehouse: 'databricks'}}), // using a utility to get a logger with context
}
const reportDatabricksTableState = handlerFactory(adapters);
// now let's use this
await reportDatabricksTableState('my-table');
# 3. How do you use them for testing?
The nice thing is, as you can imagine, it’s pretty easy to unit test. To demonstrate I’ll use vitest, which is the test library I usually use at work, but replace for jest in your head or whatever.
Because the adapters are the input and are so lightweight (no auth stuff etc like we may have in the straight-up Databricks client), we can mock their implementation and scenarios without any boilerplate.
import { describe, expect, test, vi } from 'vitest';
import { nullLogger } from './test-utils';
describe('report table state', () => {
it('if table does not exist, writes back exists: false and does not count rows', async () => {
const mockAdapters: Adapters = {
countRows: vi.fn(),
checkIfTableExists: vi.fn(() => Promise.resolve(false)),
writeTableState: vi.fn(),
logger: nullLogger(),
}
const handler = handlerFactory(mockAdapters);
await handler('foo');
expect(mockAdapters.countRows).not.toHaveBeenCalled();
expect(mockAdapters.upsertTableState).toHaveBeenCalledExactlyOnceWith('foo', { exists: false });
});
// and so on.
});
# 4. How do you use them to save on memory or API calls?
Well, today I found this out. Say you are in an implementation in Snowflake where you, for whatever reason, have already checked that your table actually exists somewhere.
import { checkIfTableExists } from 'my-snowflake-impl';
function manageSnowflakeTables(snowflakeClient: SnowflakeClient, databaseName: string) {
...
// some other functionality that ends up checking if the table exists
const tableExists = await checkIfTableOrViewExists(snowflakeClient, {tableName: myTableName, type: 'table'}); // notice that this fn takes the client as an argument! just to illustrate that there can be different implementations, doesn't have to be classes with methods
logger.info('yay, my table exists! I will do something with that information.')
...
// now we want to report back the table state.
}
If we create our adapters by implementing the method again, we’ll need to make a redundant call to Snowflake. Given that compute costs can be significant in Snowflake, this is a problem. The good news is, with our adapter pattern, we don’t actually need to do this! We can stub the method to use the results we’ve gotten previously.
import {countRows, checkIfTableOrViewExists} from 'my-snowflake-impl';
function manageSnowflakeTables(snowflakeClient: SnowflakeClient, databaseName: string) {
...
// some other functionality that ends up checking if the table exists
const tableExists = await checkIfTableOrViewExists(snowflakeClient, {tableName: myTableName, type: 'table'});
logger.info('yay, my table exists! I will do something with that information.')
...
// now we want to report back the table state.
const reportTableState = handlerFactory({
countRows: (tableName: string) => countRows(snowflakeClient, tableName), // we don't have this info yet, so let's implement the actual method
checkIfTableExists: (_) => Promise.resolve(tableExists), // BOOM! No need to use the Snowflake client again.
...
});
await reportTableState(myTableName);
}
You’d use them the exact same way as above to save on memory, for example if your adapter requires you to fetch a large piece of data (as opposed to a lil boolean). If you’ve already fetched that large piece of data and have it in memory, there’s no need to get it again.
# Conclusion
I’ve just finally looked a bit, because I didn’t want other people’s definitions to influence my own, and it looks like the adapter pattern is mostly used in Object-oriented programming. This makes sense - but I do think it has its place in functional programming as well. It allows us to abstract away the intricacies of specific implementations of a behavior in one domain or one class, to have a reusable functionality that implements predictable behavior!