All host origin patterns required for this supplier to function.
Automatically includes baseURL and, if defined, apiURL. Used by the
factory to check chrome permissions before querying.
Creates a new instance of the supplier base class. Initializes the supplier with query parameters, request limits, and abort controller. Sets up logging and default product values.
The search term to query products for
The maximum number of results to return (default: 5)
Optionalcontroller: AbortControllerAbortController instance for managing request cancellation
// Create a supplier with default limit
const supplier = new MySupplier("sodium chloride", undefined, new AbortController());
// Create a supplier with custom limit
const supplier = new MySupplier("acetone", 10, new AbortController());
// Create a supplier and handle cancellation
const controller = new AbortController();
const supplier = new MySupplier("ethanol", 5, controller);
// Later, to cancel all pending requests:
controller.abort();
Initializes the cache for the supplier. This is called after construction to ensure supplierName is set.
The cache is initialized with the supplier's name and is used to store both query results and product data. This method should be called after the supplier's name is set to ensure proper cache key generation.
class MySupplier extends SupplierBase<Product> {
constructor() {
super("acetone", 5);
// supplierName is set here
this.initCache(); // Initialize cache after supplierName is set
}
}
public initCache(
enabled: boolean = true,
doNotCacheEmptyResults: boolean = false,
cacheTtlMinutes: number = 0,
): void {
this.cache = new SupplierCache(
this.supplierName,
this.constructor.name,
enabled,
doNotCacheEmptyResults,
cacheTtlMinutes,
);
}
Applies (or clears) a runtime override for the fuzz scorer. Driven by
userSettings.fuzzScorerOverride — when the user picks a scorer in the
Advanced drawer section, SupplierFactory calls this on each instance
so the choice takes effect uniformly across every supplier.
Silently ignores unknown names so an outdated / corrupted setting can't blow up the search flow — callers fall back to the subclass default.
Name of a scorer from FUZZ_SCORERS, or undefined to
clear the override and use the subclass default.
const supplier = new MySupplier("acetone", 5, controller);
supplier.setFuzzScorerOverride("token_set_ratio");
// fuzzyFilter now uses token_set_ratio regardless of MySupplier's default
supplier.setFuzzScorerOverride(undefined);
// back to MySupplier's default
public setFuzzScorerOverride(name: string | undefined): void {
if (isFuzzScorerName(name)) {
this.fuzzScorerOverride = FUZZ_SCORERS[name];
} else {
this.fuzzScorerOverride = undefined;
}
}
ProtectedsetupPlaceholder for any setup that needs to be done before the query is made. Override this in subclasses if you need to perform setup (e.g., authentication, token fetching).
A promise that resolves when the setup is complete.
await supplier.setup();
protected async setup(): Promise<void> {}
ProtectedhttpRetrieves HTTP headers from a URL using a HEAD request. Useful for checking content types, caching headers, and other metadata without downloading the full response.
The URL to fetch headers from
Promise resolving to the response headers or void if request fails
// Basic usage
const headers = await supplier.httpGetHeaders('https://example.com/product/123');
if (headers) {
console.log('Content-Type:', headers['content-type']);
}
// With error handling
try {
const headers = await supplier.httpGetHeaders('https://example.com/product/123');
if (headers) {
console.log('Headers:', headers);
}
} catch (err) {
console.error('Failed to fetch headers:', err);
}
protected async httpGetHeaders(url: string | URL): Promise<Maybe<HeadersInit>> {
const requestObj = new Request(this.href(url), {
signal: this.controller.signal,
headers: new Headers(this.headers),
referrer: this.baseURL,
referrerPolicy: "strict-origin-when-cross-origin",
body: null,
method: "HEAD",
mode: "cors",
credentials: "include",
});
try {
const httpResponse = await this.fetch(requestObj);
return Object.fromEntries(httpResponse.headers.entries()) satisfies HeadersInit;
} catch (error: unknown) {
if (error instanceof Error && error.name === "AbortError") {
this.logger.warn("Request was aborted", { error, signal: this.controller.signal });
this.controller.abort();
} else {
this.logger.error("Error received during fetch:", {
error,
signal: this.controller.signal,
});
}
return;
}
}
ProtectedhttpSends a POST request to the given URL with the given body and headers. Handles request setup, error handling, and response caching.
The request configuration options
Promise resolving to the Response object or void if request fails
// Basic POST request
const response = await supplier.httpPost({
path: '/api/v1/products',
body: { name: 'Test Chemical' }
});
// POST with custom headers
const response = await supplier.httpPost({
path: '/api/v1/products',
body: { name: 'Test Chemical' },
headers: {
'Authorization': 'Bearer token123',
'Content-Type': 'application/json'
}
});
// POST with custom host and params
const response = await supplier.httpPost({
path: '/api/v1/products',
host: 'api.example.com',
body: { name: 'Test Chemical' },
params: { version: '2' }
});
// Error handling
try {
const response = await supplier.httpPost({ path: '/api/v1/products', body: { name: 'Test' } });
if (response && response.ok) {
const data = await response.json();
console.log('Created:', data);
}
} catch (err) {
console.error('POST failed:', err);
}
protected async httpPost({
path,
host,
body,
params,
headers,
}: RequestOptions): Promise<Maybe<Response>> {
const method = "POST";
const mode = "cors";
const referrer = this.baseURL;
const referrerPolicy = "strict-origin-when-cross-origin";
const signal = this.controller.signal;
const bodyStr = typeof body === "string" ? body : (JSON.stringify(body) ?? null);
const headersObj = new Headers({
...this.headers,
...headers,
});
const url = this.href(path, params, host);
const requestObj = new Request(url, {
signal,
headers: headersObj,
referrer,
referrerPolicy,
body: bodyStr,
method,
mode,
credentials: "include",
});
// Fetch the goods
const httpResponse = await this.fetch(requestObj);
if (!isHttpResponse(httpResponse) || !httpResponse.ok) {
const badResponse = await httpResponse.text();
this.logger.error("Invalid POST response: ", badResponse);
throw new TypeError(`Invalid POST response: ${httpResponse?.toString()}`);
}
return httpResponse;
}
ProtectedhttpSends a POST request and returns the response as a JSON object.
The parameters for the POST request.
The response from the POST request as a JSON object.
// Basic usage
const data = await supplier.httpPostJson({
path: '/api/v1/products',
body: { name: 'John' }
});
// With custom headers and error handling
try {
const data = await supplier.httpPostJson({
path: '/api/v1/products',
body: { name: 'John' },
headers: { 'Authorization': 'Bearer token123' }
});
if (data) {
console.log('Created:', data);
}
} catch (err) {
console.error('POST JSON failed:', err);
}
protected async httpPostJson({
path,
host,
body,
params,
headers,
}: RequestOptions): Promise<Maybe<JsonValue>> {
const httpResponse = await this.httpPost({ path, host, body, params, headers });
if (!isJsonResponse(httpResponse) || !httpResponse.ok) {
this.logger.error("httpPostJson| Invalid POST response: ", {
httpResponse,
path,
host,
body,
params,
headers,
});
throw new TypeError(`httpPostJson| Invalid POST response: ${httpResponse}`);
}
return await httpResponse.json();
}
ProtectedhttpSends a POST request and returns the response as a HTML string.
The request configuration options
Promise resolving to the HTML response as a string or void if request fails
TypeError - If the response is not valid HTML content
// Basic usage
const html = await supplier.httpPostHtml({
path: '/api/v1/products',
body: { name: 'John' }
});
protected async httpPostHtml({
path,
host,
body,
params,
headers,
}: RequestOptions): Promise<Maybe<string>> {
const httpResponse = await this.httpPost({ path, host, body, params, headers });
if (!isHtmlResponse(httpResponse)) {
throw new TypeError(`httpPostHtml| Invalid POST response: ${httpResponse}`);
}
return await httpResponse.text();
}
ProtectedhttpSends a GET request to the given URL with the specified options. Handles request setup, error handling, and response caching.
The request configuration options
Promise resolving to the Response object or void if request fails
// Basic GET request
const response = await supplier.httpGet({
path: '/products/search',
params: { query: 'sodium chloride' }
});
// GET with custom headers
const response = await supplier.httpGet({
path: '/products/search',
headers: { 'Accept': 'application/json' }
});
// GET with custom host
const response = await supplier.httpGet({
path: '/products/search',
host: 'api.example.com',
params: { category: 'chemicals' }
});
// Error handling
try {
const response = await supplier.httpGet({ path: '/products/search' });
if (response && response.ok) {
const data = await response.json();
console.log('Products:', data);
}
} catch (err) {
console.error('GET failed:', err);
}
protected async httpGet({
path,
params,
headers,
host,
}: RequestOptions): Promise<Maybe<Response>> {
// Check if the request has been aborted before proceeding
if (this.controller.signal.aborted) {
this.logger.warn("Request was aborted before fetch", {
signal: this.controller.signal,
});
return;
}
const headersRaw = { ...this.headers };
Object.assign(headersRaw, {
accept: [
"text/html",
"application/xhtml+xml",
"application/xml;q=0.9",
"image/avif",
"image/webp",
"image/apng",
"*/*;q=0.8",
].join(","),
...(headers ?? {}),
});
const requestObj = new Request(this.href(path, params, host), {
signal: this.controller.signal,
headers: new Headers(headersRaw),
referrer: this.baseURL,
referrerPolicy: "no-referrer",
body: null,
method: "GET",
mode: "cors",
credentials: "include",
redirect: "follow",
});
try {
// Fetch the goods
const httpResponse = await this.fetch(requestObj.url, requestObj);
const responseHeaders = Object.fromEntries(
httpResponse.headers.entries(),
) satisfies HeadersInit;
this.logger.debug("responseHeaders:", responseHeaders);
this.logger.debug("responseHeaders.location:", responseHeaders.location);
return httpResponse;
} catch (error: unknown) {
if (error instanceof Error && error.name === "AbortError") {
this.logger.warn("Request was aborted", { error, signal: this.controller.signal });
this.controller.abort();
} else {
this.logger.error("Error received during fetch:", {
error,
signal: this.controller.signal,
});
}
return;
}
}
ProtectedfuzzyFilters an array of data using fuzzy string matching to find items that closely match a query string. Uses the WRatio algorithm from fuzzball for string similarity comparison.
The search string to match against
Array of data objects to search through
Minimum match percentage (0-100) for a match to be included (default: 55)
Array of matching data objects with added fuzzy match metadata
// Example with simple string array
const products = [
{ title: "Sodium Chloride", price: 29.99 },
{ title: "Sodium Hydroxide", price: 39.99 },
{ title: "Potassium Chloride", price: 19.99 }
];
const matches = this.fuzzyFilter("sodium chloride", products);
// Returns: [
// {
// title: "Sodium Chloride",
// price: 29.99,
// _fuzz: { score: 100, idx: 0 }
// },
// {
// title: "Sodium Hydroxide",
// price: 39.99,
// _fuzz: { score: 85, idx: 1 }
// }
// ]
// Example with custom minMatchPercentage
const strictMatches = this.fuzzyFilter("sodium chloride", products, 90);
// Returns only exact matches with score >= 90
// Example with different data structure
const chemicals = [
{ name: "NaCl", formula: "Sodium Chloride" },
{ name: "NaOH", formula: "Sodium Hydroxide" }
];
// Override titleSelector to use formula field
this.titleSelector = (data) => data.formula;
const formulaMatches = this.fuzzyFilter("sodium chloride", chemicals);
protected fuzzyFilter<X>(
query: string,
data: X[],
minMatchPercentage: number = this.minMatchPercentage,
): X[] {
// User's Advanced-settings override wins over the subclass default.
const activeScorer = this.fuzzScorerOverride ?? this.fuzzScorer;
// console.log(
// `[fuzzyFilter] ${this.supplierName} query="${query}" — scorer comparison (cutoff=${minMatchPercentage})`,
// );
if (IS_DEV_BUILD) {
this.showFuzzScorerComparisonTable(query, data);
}
const results = extract(query, data, {
scorer: activeScorer,
processor: this.titleSelector,
cutoff: minMatchPercentage,
sortBySimilarity: true,
}).reduce<FuzzyMatchResult<X>[]>((acc, [obj, score, idx]) => {
if (score < minMatchPercentage) {
this.logger.debug("fuzzyFilter: score below minimum match percentage, excluding product", {
product: obj,
score,
idx,
minMatchPercentage: minMatchPercentage,
});
return acc;
}
// eslint-disable-next-line @typescript-eslint/naming-convention
acc[idx] = Object.assign(obj, { _fuzz: { score, idx }, matchPercentage: score });
return acc;
}, []);
this.logger.debug("[fuzzyFilter]", {
supplierName: this.supplierName,
query,
minMatchPercentage,
activeScorer,
results,
});
// Get rid of any empty items that didn't match closely enough
return results.filter((item) => !!item);
}
ProtectedhttpMakes an HTTP GET request and returns the response as a string. Handles request configuration, error handling, and HTML parsing.
The request configuration options
Promise resolving to the HTML response as a string or void if request fails
TypeError - If the response is not valid HTML content
// Basic GET request
const html = await this.httpGetHtml({
path: "/api/products",
params: { search: "sodium" }
});
// GET request with custom headers
const html = await this.httpGetHtml({
path: "/api/products",
headers: {
"Authorization": "Bearer token123",
"Accept": "text/html"
}
});
// GET request with custom host
const html = await this.httpGetHtml({
path: "/products",
host: "api.supplier.com",
params: { limit: 10 }
});
protected async httpGetHtml({
path,
params,
headers,
host,
}: RequestOptions): Promise<Maybe<string>> {
const httpResponse = await this.httpGet({ path, params, headers, host });
if (!isHtmlResponse(httpResponse)) {
throw new TypeError(`httpGetHtml| Invalid GET response: ${httpResponse}`);
}
return await httpResponse.text();
}
ProtectedhttpMakes an HTTP GET request and returns the response as parsed JSON. Handles request configuration, error handling, and JSON parsing.
The request configuration options
Promise resolving to the parsed JSON response or void if request fails
TypeError - If the response is not valid JSON content
// Basic GET request
const data = await supplier.httpGetJson({ path: '/api/products', params: { search: 'sodium' } });
// GET request with custom headers
const data = await supplier.httpGetJson({
path: '/api/products',
headers: {
'Authorization': 'Bearer token123',
'Accept': 'application/json'
}
});
// GET request with custom host
const data = await supplier.httpGetJson({
path: '/products',
host: 'api.supplier.com',
params: { limit: 10 }
});
// Error handling
try {
const data = await supplier.httpGetJson({ path: '/api/products' });
if (data) {
console.log('Products:', data);
}
} catch (error) {
console.error('Failed to fetch products:', error);
}
protected async httpGetJson({
path,
params,
headers,
host,
}: RequestOptions): Promise<Maybe<JsonValue>> {
const httpRequest = await this.httpGet({ path, params, headers, host });
if (!isJsonResponse(httpRequest)) {
const badResponse = isHttpResponse(httpRequest) ? await httpRequest.text() : undefined;
this.logger.error("Invalid HTTP GET JSON response:", {
badResponse,
httpRequest,
path,
params,
headers,
host,
});
return;
}
return await httpRequest.json();
}
ProtectedqueryExecutes a product search query with caching support. First checks the cache for existing results, then falls back to the actual query if needed. The limit parameter is only used for the actual query and doesn't affect caching.
The search term to query products for
The maximum number of results to return (defaults to instance limit)
Promise resolving to array of product builders or void if search fails
// Basic usage with default limit
const results = await supplier.queryProductsWithCache("acetone");
if (results) {
console.log(`Found ${results.length} products`);
}
// With custom limit
const results = await supplier.queryProductsWithCache("acetone", 10);
if (results) {
for (const builder of results) {
const product = await builder.build();
console.log(product.title, product.price);
}
}
protected async queryProductsWithCache(
query: string,
limit: number = this.limit,
): Promise<ProductBuilder<T>[] | void> {
// Check cache first (processed product data)
this.logger.debug(
"queryProductsWithCache: called for",
this.supplierName,
"query:",
query,
"limit:",
limit,
);
const key = this.cache.generateCacheKey(query);
const cached = await this.cache.getCachedQueryEntry(key);
this.logger.debug("queryProductsWithCache: cache hit:", !!cached, "key:", key);
if (cached) {
// If the cached limit is less than the requested limit, invalidate the cache
if (
typeof cached.__cacheMetadata.limit === "number" &&
cached.__cacheMetadata.limit < limit
) {
this.logger.debug("Invalidating query cache due to insufficient limit", {
cachedLimit: cached.__cacheMetadata.limit,
requestedLimit: limit,
});
await deleteSupplierQueryCacheEntry(key);
} else {
this.logger.debug("Returning cached query results");
// Re-initialize product builders from cached processed data
return ProductBuilder.createFromCache<T>(this.baseURL, cached.data.slice(0, limit));
}
}
// If not in cache, perform the actual query. Run setup first so any
// subclass state it mutates (headers, localStorage, tokens, etc.) is
// in place before `queryProducts` reads it. Memoized, so this is cheap
// on repeat calls within the same supplier instance.
await this.ensureSetup();
const results = await this.queryProducts(query, limit);
if (results) {
// Store processed results in cache (dumped/serialized form) and the limit used
await this.cache.cacheQueryResults(
query,
results.map((b) => b.dump()),
limit,
);
}
return results;
}
Executes the supplier's search query and returns the results. This method will execute all results concurrently (to the limits set in the supplier class), and resolve to an array of product objects.
Promise resolving to an array of products
This method is used to execute the supplier's search query and return the results.
public async *execute(): AsyncGenerator<T, void, undefined> {
// setup() is not called eagerly here — it's run lazily from the
// phase-boundary gates inside `queryProductsWithCache` and
// `getProductData` / `getProductDataWithCache`. A fully cached search
// never reaches those gates, so setup's token/cookie/permission
// requests are skipped entirely.
// Snapshot the user's ignore list once per search. Any product whose
// exclusion key matches an entry here is dropped before the detail phase
// runs (see the filter after queryProductsWithCache below).
this.excludedProductKeys = await loadExcludedProductKeys();
// Over-fetch by the number of previously-ignored products belonging to
// this supplier so that, in the worst case where every ignored product
// appears in the top of the query result set, we still end up with
// `this.limit` survivors after filtering. The queryProductsWithCache
// cache invalidates itself when the requested limit exceeds the cached
// limit, so this is safe.
const excludedForSupplier = await countExcludedProductsForSupplier(this.supplierName);
const fetchLimit = this.limit + excludedForSupplier;
incrementSearchQueryCount(this.supplierName);
const results = await this.queryProductsWithCache(this.query, fetchLimit);
if (!results || results.length === 0) {
this.logger.log(`No query results found`);
return;
}
// Drop any products the user has ignored, then slice back down to the
// user-visible limit. Uses the same key shape as getProductData so
// whichever side catches the exclusion first, the check is consistent.
const survivors: ProductBuilder<T>[] = [];
for (const builder of results) {
if (survivors.length >= this.limit) break;
const rawUrl = builder.get("url");
if (typeof rawUrl !== "string") {
survivors.push(builder);
continue;
}
const exclusionUrl = this.href(rawUrl);
const exclusionKey = getProductExclusionKey(exclusionUrl, this.supplierName);
if (this.excludedProductKeys.has(exclusionKey)) {
this.logger.debug("Skipping excluded product (pre-detail)", {
url: rawUrl,
exclusionUrl,
exclusionKey,
});
continue;
}
survivors.push(builder);
}
this.products = survivors;
const queue = new Queue(this.maxConcurrentRequests, this.minConcurrentCycle);
// Create an array of promises, each yielding a product as soon as it's ready
const tasks = this.products.map((product) =>
queue.run(async () => {
try {
this.logger.debug(`Product data for ${this.supplierName}:`, product);
const builder = await this.getProductData(product);
if (!builder) return;
this.logger.debug(`Builder data for ${this.supplierName}:`, builder);
const finished = await this.finishProduct(builder);
this.logger.debug(`Finished product data for ${this.supplierName}:`, finished);
if (finished) {
return finished;
}
} catch (e: unknown) {
this.logger.error("Error processing product", { error: e, product });
incrementParseError(this.supplierName);
}
}),
);
// As each promise resolves, yield the product
const resultsSet = new Set(tasks);
while (resultsSet.size > 0) {
const finished = await Promise.race(resultsSet);
// Remove the finished promise from the set
for (const t of resultsSet) {
if ((await Promise.resolve(t)) === finished) {
resultsSet.delete(t);
break;
}
}
if (finished) {
yield finished;
}
}
}
ProtectedfinishFinalizes a partial product by adding computed properties and validating the result. This method:
The ProductBuilder instance containing the partial product to finalize
Promise resolving to a complete Product object or void if validation fails
// Example with a valid partial product
const builder = new ProductBuilder<Product>(this.baseURL);
builder
.setBasicInfo("Sodium Chloride", "/products/nacl", "ChemSupplier")
.setPricing(29.99, "USD", "$")
.setQuantity(500, "g");
const finishedProduct = await this.finishProduct(builder);
if (finishedProduct) {
console.log("Finalized product:", {
title: finishedProduct.title,
price: finishedProduct.price,
quantity: finishedProduct.quantity,
uom: finishedProduct.uom,
usdPrice: finishedProduct.usdPrice,
baseQuantity: finishedProduct.baseQuantity
});
}
// Example with an invalid partial product
const invalidBuilder = new ProductBuilder<Product>(this.baseURL);
invalidBuilder.setBasicInfo("Sodium Chloride", "/products/nacl", "ChemSupplier");
// Missing required fields
const invalidProduct = await this.finishProduct(invalidBuilder);
if (!invalidProduct) {
console.log("Failed to finalize product - missing required fields");
}
protected async finishProduct(product: ProductBuilder<T>): Promise<Maybe<T>> {
if (!isMinimalProduct(product.dump())) {
this.logger.warn("Unable to finish product - Minimum data not set", { product });
return;
}
// Set the country and shipping scope of the supplier
// have different restrictions on different products or countries.
product.setSupplierCountry(this.country);
product.setSupplierShipping(this.shipping);
if (this.paymentMethods.length > 0) {
product.setSupplierPaymentMethods(this.paymentMethods);
}
const built = await product.build();
return built;
}
ProtectedhrefTakes in either a relative or absolute URL and returns an absolute URL. This is useful for when you aren't sure if the link (retrieved from parsed text, a setting, an element, an anchor value, etc) is absolute or not. Using relative links will result in http://chrome-extension://... being added to the link.
URL object or string
Optionalparams: Maybe<RequestParams>The parameters to add to the URL.
Optionalhost: stringThe host to use for overrides (eg: needing to call a different host for an API)
absolute URL
this.href('/some/path')
// https://supplier_base_url.com/some/path
this.href('https://supplier_base_url.com/some/path', null, 'another_host.com')
// https://another_host.com/some/path
this.href('/some/path', { a: 'b', c: 'd' }, 'another_host.com')
// http://another_host.com/some/path?a=b&c=d
this.href('https://supplier_base_url.com/some/path')
// https://supplier_base_url.com/some/path
this.href(new URL('https://supplier_base_url.com/some/path'))
// https://supplier_base_url.com/some/path
this.href('/some/path', { a: 'b', c: 'd' })
// https://supplier_base_url.com/some/path?a=b&c=d
this.href('https://supplier_base_url.com/some/path', new URLSearchParams({ a: 'b', c: 'd' }))
// https://supplier_base_url.com/some/path?a=b&c=d
protected href(path: string | URL, params?: Maybe<RequestParams>, host?: string): string {
const href = new URL(path, this.baseURL);
if (host) {
href.host = host;
}
if (params && Object.keys(params).length > 0) {
href.search = new URLSearchParams(
Object.entries(params).reduce<QueryParams>((acc, [key, value]) => {
acc[key] = String(value);
return acc;
}, {}),
).toString();
}
return href.toString();
}
ProtectedgetRetrieves product data with caching support. Similar to getProductData but allows for additional parameters to be included in the cache key.
The ProductBuilder instance to get data for
The function to use for fetching product data
Optionalparams: QueryParamsOptional parameters to include in the cache key
Promise resolving to the updated ProductBuilder or void if fetch fails
const builder = new ProductBuilder<Product>(this.baseURL);
builder.setBasicInfo("Acetone", "/products/acetone", "ChemSupplier");
// Use custom fetcher with additional params
const updatedBuilder = await supplier.getProductDataWithCache(
builder,
async (b) => {
// Custom fetching logic
return b;
},
{ version: "2.0" }
);
protected async getProductDataWithCache(
product: ProductBuilder<T>,
fetcher: (builder: ProductBuilder<T>) => Promise<ProductBuilder<T> | void>,
params?: QueryParams,
): Promise<ProductBuilder<T> | void> {
const url = product.get("url");
if (typeof url !== "string") {
this.logger.error("Invalid URL in product:", { url });
return undefined;
}
// See getProductData above: normalize only for the exclusion key so it
// lines up with the absolute URL the UI context menu stores.
const exclusionUrl = this.href(url);
const shouldExclude = await shouldExcludeProduct(exclusionUrl, this.supplierName);
if (shouldExclude) {
this.logger.debug("Skipping excluded product", {
url,
exclusionUrl,
supplierName: this.supplierName,
});
return undefined;
}
const cacheKey = this.cache.getProductDataCacheKey(url, params);
this.logger.debug("[SupplierBase] Product detail cache key:", cacheKey, "for url:", url);
try {
const cachedData = await this.cache.getCachedProductData(cacheKey);
if (cachedData) {
// Safe: cache only stores values previously produced by ProductBuilder<T>.dump(),
// so the round-tripped shape is structurally a Partial<T>.
product.setData(cachedData as Partial<T>);
return product;
}
// Cache miss: run setup (memoized) so any state subclasses rely on is
// ready before the fetcher reads it, then call the fetcher.
await this.ensureSetup();
let resultBuilder: ProductBuilder<T> | void = undefined;
try {
resultBuilder = await fetcher(product);
} catch (err: unknown) {
this.logger.error("Error in product detail fetcher:", err);
incrementParseError(this.supplierName);
return undefined;
}
if (resultBuilder) {
incrementProductCount(this.supplierName);
await this.cache.cacheProductData(cacheKey, resultBuilder.dump());
}
return resultBuilder;
} catch (outerErr: unknown) {
this.logger.error("Error in getProductDataWithCache:", outerErr);
incrementParseError(this.supplierName);
return undefined;
}
}
ProtectedgroupGroups variants of a product by their title
Array of product listings from search results
Array of product listings with grouped variants
Create a generic method for this, the same method is used in Synthetika and could be of use with LoudWolf.
const results = await this.queryProducts("sodium chloride");
const grouped = this.groupVariants(results);
// grouped is an array of product listings with grouped variants
protected groupVariants<R>(data: R[]): R[] {
const variants: GroupedItem<R>[] = data
.map((item) => {
const title = this.titleSelector(item);
if (!title) {
this.logger.error("No title found in product:", { item });
return undefined;
}
const groupId = stripQuantityFromString(title.replace(/(?<=\d{1,3})\s(?=\d{3})/g, ""));
const groupIdWithoutSpaces = groupId.replace(/[\s-]/g, "");
return { ...item, groupId: groupIdWithoutSpaces };
})
.filter((item): item is GroupedItem<R> => item !== undefined);
const products = Object.groupBy(variants, (item) => item.groupId);
return Object.values(products)
.filter((product): product is GroupedItem<R>[] => product !== undefined)
.map((product) => {
const main = product.splice(0, 1)[0];
// eslint-disable-next-line @typescript-eslint/no-unused-vars
const { groupId, ...newObject } = main;
newObject.variants = product as GroupedItem<R>["variants"];
return newObject;
})
.filter((item): item is GroupedItem<R> => item !== undefined);
}
ProtectedfetchInternal fetch method with request counting and decorator. Tracks request count and enforces hard limits on HTTP requests.
Arguments to pass to fetchDecorator (usually a Request or URL and options)
The response from the fetchDecorator
Error if request count exceeds hard limit
// Example usage inside a subclass:
const response = await this.fetch(new Request('https://example.com'));
if (response.ok) {
const data = await response.json();
console.log(data);
}
// With custom request options
const response = await this.fetch(
new Request('https://example.com', {
headers: { 'Accept': 'application/json' }
})
);
protected async fetch(
...args: Parameters<typeof fetchDecorator>
): Promise<FetchDecoratorResponse> {
const [input] = args;
this.logger.debug(`Fetching: ${input}`);
// One initial attempt plus up to `challengeRetryLimit` retries. A 403 from
// a WAF cookie handshake plants a cookie on the first hit (stored because
// credentials:"include"); the retry sends it back and usually passes.
const maxAttempts = 1 + Math.max(0, this.challengeRetryLimit);
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
// Each attempt is a real network request, so it counts toward the hard
// limit. For non-retrying suppliers (maxAttempts === 1) this is
// identical to the previous single increment.
this.requestCount++;
if (this.requestCount > this.httpRequestHardLimit) {
this.logger.warn("Request count exceeded hard limit", { requestCount: this.requestCount });
incrementFailure(this.supplierName);
throw new Error("Request count exceeded hard limit");
}
try {
const response = await fetchDecorator(...args);
this.logger.debug(`Response Status: ${response.status}`);
this.logger.debug("response hash:", response.requestHash);
if (typeof response.data === "string" && response.data?.length === 0) {
throw new EmptyResponseError(`Invalid response: ${response.data}`);
}
incrementSuccess(this.supplierName);
return response;
} catch (error: unknown) {
if (this.shouldRetryChallenge(error) && attempt < maxAttempts) {
this.logger.warn("Retrying after 403 (WAF cookie handshake)", {
attempt,
maxAttempts,
input,
});
await new Promise((resolve) => setTimeout(resolve, this.challengeRetryDelayMs));
continue;
}
incrementFailure(this.supplierName);
throw error;
}
}
// Unreachable: the loop always returns on success or throws on the final
// failed attempt. Present only to satisfy the return-type checker.
throw new Error("fetch: exhausted retries without resolving");
}
ProtectednormalizeNormalizes a Shopware-rendered price string (German locale) into a
format that parsePrice can consume unambiguously.
Shopware serves numbers as 2.449,00 $ (period = thousands,
comma = decimal) regardless of the selected currency, so the $ symbol
and the German digit formatting disagree. Strip the footnote *, the
non-breaking space, the thousands ., and swap the decimal , to ..
Raw price text from the DOM (e.g. "2.449,00\u00A0$ *")
Normalized string safe for parsePrice (e.g. "2449.00 $")
this.normalizePriceText("12,25\u00A0$ *"); // "12.25 $"
this.normalizePriceText("2.449,00\u00A0$"); // "2449.00 $"
protected normalizePriceText(text: string): string {
return text
.replace(/\u00A0/g, " ")
.replace(/\*/g, "")
.replace(/\./g, "")
.replace(",", ".")
.trim();
}
ProtectedqueryQueries S3 Chemicals products based on a search string.
Makes a GET request to the Shopware /search endpoint asking for the
largest allowed page size (n=48) and parses the HTML response to extract
basic product information from each .product--box card.
The search term to query products for
The maximum number of results to query for
Promise resolving to an array of ProductBuilder instances or void if search fails
const supplier = new SupplierS3Chemicals("eosin", 10, new AbortController());
const results = await supplier.queryProducts("eosin");
if (results) {
console.log(`Found ${results.length} products`);
console.log("First product:", results[0].get("title"));
}
protected async queryProducts(
query: string,
limit: number = this.limit,
): Promise<ProductBuilder<Product>[] | void> {
const searchRequest = await this.httpGetHtml({
path: "/search",
params: {
sSearch: encodeURIComponent(query),
p: 1,
// Shopware's per-page size is restricted to one of {12, 24, 36, 48}.
n: 48,
},
});
if (!searchRequest) {
this.logger.error("No search response", { query });
return;
}
this.logger.log("Received search response", { query, searchRequest });
const fuzzResults = this.fuzzHtmlResponse(query, searchRequest);
this.logger.info("fuzzResults:", { query, searchRequest, fuzzResults });
const builders = this.initProductBuilders(fuzzResults.slice(0, limit));
this.logger.info("builders:", { query, searchRequest, fuzzResults, builders });
return builders;
}
ProtectedfuzzParses HTML response and performs fuzzy filtering on product elements.
Creates a DOM from the HTML response, selects div.product--box.box--basic
cards from the search listing, and applies fuzzy filtering against the
card titles returned by titleSelector.
The search term to filter products by
The HTML response string containing product listings
Array of DOM Elements that match the fuzzy search criteria
const html = await this.httpGetHtml({ path: "/search", params: { sSearch: "eosin" } });
if (html) {
const matches = this.fuzzHtmlResponse("eosin", html);
console.log(`Found ${matches.length} matching product cards`);
}
protected fuzzHtmlResponse(query: string, response: string): Element[] {
const parsedHTML = createDOM(response);
if (!parsedHTML || parsedHTML === null) {
throw new Error("No data found when loading HTML");
}
const productContainers = parsedHTML.querySelectorAll("div.product--box.box--basic");
if (!productContainers || productContainers.length === 0) {
this.logger.log("No products found", { query, response, parsedHTML, productContainers });
return [];
}
return this.fuzzyFilter<Element>(query, Array.from(productContainers));
}
ProtectedinitInitialize product builders from S3 Chemicals HTML search response cards.
Transforms each div.product--box card into a ProductBuilder instance with
title, URL, ordernumber (as ID), listing price, description, and quantity.
Cards that are missing a title or URL are skipped silently via mapDefined.
Array of DOM Elements containing product cards
Array of ProductBuilder instances initialized with card data
const fuzz = this.fuzzHtmlResponse("eosin", html);
const builders = this.initProductBuilders(fuzz);
// Each builder contains parsed product data from the listing card
for (const builder of builders) {
console.log(builder.get("title"), builder.get("price"));
}
protected initProductBuilders(elements: Element[]): ProductBuilder<Product>[] {
this.logger.info("initProductBuilders elements:", { elements });
return mapDefined(elements, (element: Element) => {
const builder = new ProductBuilder<Product>(this.baseURL);
const anchor = element.querySelector("a.product--title");
const href = anchor?.getAttribute("href");
const title = anchor?.getAttribute("title")?.trim() || anchor?.textContent?.trim();
if (!anchor || !href || !title) {
this.logger.error("No title/href for product", { element });
return;
}
// Search results already return absolute URLs; `new URL` with a base is
// defensive and a no-op when `href` is absolute.
const url = new URL(href, this.baseURL);
// Shopware exposes the ordernumber on the wrapper (e.g. "S100210").
// A small number of grouped-variant parents may omit it; we fall back
// to the numeric productID meta in `getProductData` when that happens.
const ordernumber = element.getAttribute("data-ordernumber") ?? undefined;
// Shopware renders prices in German locale (`2.449,00 $`) even when
// the currency is USD — normalize to dot-decimal before `parsePrice`,
// otherwise price-parser mis-reads the comma-decimal form (e.g.
// `"12,25 $"` comes out as $0.12).
const priceText = this.normalizePriceText(
element.querySelector("div.product--price span.price--default")?.textContent ?? "",
);
const price = parsePrice(priceText);
if (price !== undefined) {
builder.setPricing(price.price, price.currencyCode, price.currencySymbol);
}
const description = element
.querySelector("div.product--description")
?.textContent?.trim()
.replace(/\s+/g, " ");
if (description) {
builder.setDescription(description);
}
const qtyText = element
.querySelector("div.price--unit span.is--nowrap")
?.textContent?.trim();
if (qtyText) {
const qty = parseQuantity(qtyText);
if (qty) {
builder.setQuantity(qty);
}
}
builder.setBasicInfo(title, url.toString(), this.supplierName);
if (ordernumber) {
builder.setID(ordernumber);
}
return builder;
});
}
ProtectedvariantReduces a variant's quantity + uom to a single scalar so an array of variants can be sorted ascending by chemical content regardless of unit. Mass units collapse to milligrams and volume units collapse to millilitres — the two families aren't comparable to each other, but S3 variants of one product always share a family, so a single ranking is enough. Unknown/missing units sort last.
Canonical magnitude (mg or mL) or +Infinity if unknown
this.variantSortRank({ quantity: 25, uom: "g" }); // 25000
this.variantSortRank({ quantity: 1, uom: "kg" }); // 1000000
this.variantSortRank({ quantity: 500, uom: "ml" }); // 500
protected variantSortRank(v: Partial<Variant>): number {
if (v.quantity === undefined) return Number.POSITIVE_INFINITY;
const uom = v.uom?.toLowerCase() ?? "";
const multipliers: Record<string, number> = {
mg: 1,
g: 1000,
kg: 1_000_000,
t: 1_000_000_000,
ml: 1,
cl: 10,
dl: 100,
l: 1000,
};
const multiplier = multipliers[uom];
if (multiplier === undefined) return Number.POSITIVE_INFINITY;
return v.quantity * multiplier;
}
ProtectedparseExtracts the variant-selector state from a parsed detail block.
Shopware renders variants as a <select data-ajax-select-variants="true">
where the name attribute (e.g. group[12]) becomes the GET key that
switches the displayed variant. Returns undefined when the page has
no variant select (single-variant products).
The variant group data, or undefined if no select is present
const dom = createDOM(await this.httpGetHtml(...));
const group = this.parseVariantGroup(dom.querySelector("div.product--details")!);
// group = { selectName: "group[12]", options: [{ value: "438", label: "50g", selected: true }, ...] }
protected parseVariantGroup(details: Element | Document): Maybe<{
selectName: string;
options: Array<{ value: string; label: string; selected: boolean }>;
}> {
const select = details.querySelector('select[data-ajax-select-variants="true"]');
const selectName = select?.getAttribute("name");
if (!select || !selectName) {
return undefined;
}
const options = Array.from(select.querySelectorAll("option")).flatMap((opt) => {
const value = opt.getAttribute("value");
if (!value) return [];
return [
{
value,
label: opt.textContent?.trim() ?? "",
selected: opt.hasAttribute("selected"),
},
];
});
if (options.length === 0) return undefined;
return { selectName, options };
}
ProtectedbuildBuilds a variant-specific URL by appending Shopware's variant group
GET parameter onto the product's base URL. Used both as the url
value stored on each Variant and as an ergonomic breadcrumb.
The product's canonical URL (no query string)
The name attribute of the variant select (e.g. group[12])
The chosen option's value attribute (e.g. 438)
The variant URL with the group parameter appended
this.buildVariantUrl(
"https://shop.es-drei.de/farbstoffe-indikatoren/12285/allurarot-ac",
"group[12]",
"438",
);
// "https://shop.es-drei.de/farbstoffe-indikatoren/12285/allurarot-ac?group%5B12%5D=438"
protected buildVariantUrl(baseUrl: string, selectName: string, value: string): string {
const url = new URL(baseUrl);
url.searchParams.set(selectName, value);
return url.toString();
}
ProtectedextractExtracts a single variant's data from a parsed detail block.
Reads the schema.org meta tags on the page (price, currency, weight,
productID, SKU) plus the visible price text (for the currency symbol)
and returns a Partial<Variant> suitable for addVariant/setVariants.
parseVariantGroup already covers option labels; the label arg is
stored as the variant's title so users can tell sizes apart.
The .product--details DOM block for this variant
Variant-specific URL (from buildVariantUrl)
Optionallabel: stringOption label from the select (e.g. "50g")
Partial variant data extracted from the DOM
const variant = this.extractVariantData(details, variantUrl, "50g");
// { title: "50g", price: 12.25, currencyCode: "USD", currencySymbol: "$",
// quantity: 50, uom: "g", id: "89624", sku: "S1002100.1", url: "..." }
protected extractVariantData(
details: Element | Document,
url: string,
label?: string,
): Partial<Variant> {
const variant: Partial<Variant> = { url };
if (label) {
variant.title = label;
}
const priceContent = details.querySelector('meta[itemprop="price"]')?.getAttribute("content");
if (priceContent) {
const normalized = Number(this.normalizePriceText(priceContent));
if (!Number.isNaN(normalized)) {
variant.price = normalized;
}
}
const currencyCode = details
.querySelector('meta[itemprop="priceCurrency"]')
?.getAttribute("content");
if (currencyCode) {
variant.currencyCode = currencyCode as CurrencyCode;
}
const visiblePrice = details.querySelector("span.price--content")?.textContent;
if (visiblePrice) {
const parsed = parsePrice(this.normalizePriceText(visiblePrice));
if (parsed?.currencySymbol) {
variant.currencySymbol = parsed.currencySymbol;
}
}
// The option label ("25g", "100g", …) is the chemical content size.
// We deliberately ignore `meta[itemprop="weight"]` because Shopware
// reports the *shipping* weight there (bottle + packaging), which made
// every variant collapse to ~1 kg in practice. Fall back to the
// "Inhalt:" line inside `div.price--unit` for single-variant products
// that have no option label.
const qtyFromLabel = label ? parseQuantity(label) : undefined;
if (qtyFromLabel) {
variant.quantity = qtyFromLabel.quantity;
variant.uom = qtyFromLabel.uom;
} else {
const unitText = details
.querySelector("div.price--unit")
?.textContent?.replace(/\s+/g, " ")
.trim();
if (unitText) {
const cleaned = unitText.replace(/inhalt\s*:/i, "").split("(")[0].trim();
const qtyFromUnit = parseQuantity(cleaned);
if (qtyFromUnit) {
variant.quantity = qtyFromUnit.quantity;
variant.uom = qtyFromUnit.uom;
}
}
}
const productID = details
.querySelector('meta[itemprop="productID"]')
?.getAttribute("content");
if (productID) {
variant.id = productID;
}
const sku = details.querySelector('span[itemprop="sku"]')?.textContent?.trim();
if (sku) {
variant.sku = sku;
}
return variant;
}
ProtectedgetTransforms a partial product item into a complete Product object.
Fetches the product detail page with template=ajax, then iterates
every option in the Shopware variant select (appending the
group[N]=value GET parameter for each) to collect every size the
product is sold in. Common fields (description, CAS, availability)
come from the initial fetch; the top-level builder price/quantity/ID
reflects the smallest variant (lowest quantity, ties broken by price).
ProductBuilder to enrich with detail-page data
Promise resolving to the enriched ProductBuilder or void if the fetch fails
const builder = new ProductBuilder<Product>("https://shop.es-drei.de");
builder.setBasicInfo("Allurarot AC", "https://shop.es-drei.de/farbstoffe-indikatoren/12285/allurarot-ac", "S3 Chemicals");
const enriched = await supplier.getProductData(builder);
// enriched.get("price") === 2.99 // smallest (5g) variant
// enriched.get("variants")?.length === 5
protected async getProductData(
product: ProductBuilder<Product>,
): Promise<ProductBuilder<Product> | void> {
return this.getProductDataWithCache(product, async (builder) => {
this.logger.debug("Querying data for partialproduct", { builder });
if (typeof builder === "undefined") {
this.logger.error("No products to get data for", { builder });
return;
}
const baseUrl = builder.get("url");
// `template=ajax` returns the same `.product--details` block without
// the surrounding layout — roughly a 95% payload reduction per request.
const initialResponse = await this.httpGetHtml({
path: baseUrl,
params: { template: "ajax" },
});
if (!initialResponse) {
this.logger.warn("No product response", { builder });
return;
}
const initialDom = createDOM(initialResponse);
const initialDetails = initialDom.querySelector("div.product--details") ?? initialDom;
// --- Common (product-wide) data from the initial fetch -------------
const detailDescription = initialDetails
.querySelector('div.product--description[itemprop="description"]')
?.textContent?.trim()
.replace(/\s+/g, " ");
if (detailDescription) {
builder.setDescription(detailDescription);
}
const availabilityHref = initialDetails
.querySelector('link[itemprop="availability"]')
?.getAttribute("href");
if (availabilityHref) {
const token = availabilityHref.split("/").pop();
if (token) {
builder.setAvailability(token);
}
}
const titleText =
initialDetails.querySelector('h1.product--title[itemprop="name"]')?.textContent?.trim() ??
"";
const cas = firstMap(
(p) => findCAS(p),
[titleText, detailDescription ?? "", builder.get("description") ?? ""],
);
if (isCAS(cas)) {
builder.setCAS(cas);
}
// --- Variant enumeration -------------------------------------------
const group = this.parseVariantGroup(initialDetails);
const selectedOption = group?.options.find((o) => o.selected) ?? group?.options[0];
const initialVariantUrl =
group && selectedOption
? this.buildVariantUrl(baseUrl, group.selectName, selectedOption.value)
: baseUrl;
const variants: Partial<Variant>[] = [
this.extractVariantData(initialDetails, initialVariantUrl, selectedOption?.label),
];
if (group && group.options.length > 1) {
const others = group.options.filter((o) => o.value !== selectedOption?.value);
const fetched = await Promise.all(
others.map(async (opt) => {
const html = await this.httpGetHtml({
path: baseUrl,
params: { template: "ajax", [group.selectName]: opt.value },
});
if (!html) return undefined;
const dom = createDOM(html);
const details = dom.querySelector("div.product--details") ?? dom;
return this.extractVariantData(
details,
this.buildVariantUrl(baseUrl, group.selectName, opt.value),
opt.label,
);
}),
);
for (const v of fetched) {
if (v) variants.push(v);
}
}
// --- Sort variants ascending by chemical quantity ------------------
// `parseQuantity` returns normalized units (1000g → 1kg, 500mg → 0.5g),
// so a raw numeric sort would put "1 kg" before "25 g". Convert each
// variant's quantity to a canonical magnitude (milligrams for mass,
// millilitres for volume) before comparing. Missing values sort last.
variants.sort((a, b) => {
const ra = this.variantSortRank(a);
const rb = this.variantSortRank(b);
if (ra !== rb) return ra - rb;
const pa = a.price ?? Number.POSITIVE_INFINITY;
const pb = b.price ?? Number.POSITIVE_INFINITY;
return pa - pb;
});
// --- Promote the smallest variant to parent-level fields -----------
const primary = variants[0];
if (primary) {
if (primary.price !== undefined) builder.setPrice(primary.price);
if (primary.currencyCode) builder.setCurrencyCode(primary.currencyCode);
if (primary.currencySymbol) builder.setCurrencySymbol(primary.currencySymbol);
if (primary.quantity !== undefined && primary.uom) {
builder.setQuantity(primary.quantity, primary.uom);
}
if (primary.sku !== undefined) builder.setSku(String(primary.sku));
// Only overwrite the parent ID if the listing card didn't give us
// the short ordernumber (which is the cleaner identifier).
if (!builder.get("id") && primary.id !== undefined) {
builder.setID(primary.id);
}
}
if (variants.length > 1) {
builder.setVariants(variants);
}
this.logger.debug("product", builder);
return builder;
});
}
ProtectedtitleExtracts the product title from a search card DOM Element.
Prefers the title attribute on a.product--title (guaranteed clean)
and falls back to the anchor's textContent when missing.
The DOM Element containing the product card
The product title, or undefined if the anchor/title is missing
const card = document.querySelector("div.product--box.box--basic");
if (card) {
const title = this.titleSelector(card);
console.log("Product title:", title);
// Output: "Allurarot AC"
}
protected titleSelector(data: Element): Maybe<string> {
if (!data) {
this.logger.error("No data for product", { data });
return undefined;
}
const anchor = data.querySelector("a.product--title");
const title = anchor?.getAttribute("title")?.trim() || anchor?.textContent?.trim();
if (!title) {
this.logger.error("No title for product", { data });
return undefined;
}
return title;
}
Protected OptionalfuzzRuntime override resolved from userSettings.fuzzScorerOverride. When
set, fuzzyFilter uses this instead of this.fuzzScorer. Undefined
(the default) means "use whatever the supplier class picked". Mutated
by setFuzzScorerOverride so it can't be readonly.
Protected Optional ReadonlyapiOptional external API hostname used by some suppliers (e.g., Typesense,
Searchanise). When set, automatically included in requiredHosts for
permission checks.
ProtectedqueryString to query for (product name, CAS, etc.). The search term that will be used to find products. Set during construction and used throughout the supplier's lifecycle.
ProtectedbaseThe base search parameters that are always included in search requests. These parameters are merged with any additional search parameters when making requests to the supplier's API.
class MySupplier extends SupplierBase<Product> {
constructor() {
super();
this.baseSearchParams = {
format: "json",
version: "2.0"
};
}
}
protected baseSearchParams: Record<string, string | number> = {};
ProtectedcontrollerThe AbortController instance used to manage and cancel ongoing requests. This allows for cancellation of in-flight requests when needed, such as when a new search is started or the supplier is disposed.
const controller = new AbortController();
const supplier = new MySupplier("acetone", 5, controller);
// Later, to cancel all pending requests:
controller.abort();
protected controller: AbortController;
ProtectedlimitThe maximum number of results to return for a search query. This is not a limit on HTTP requests, but rather the number of products that will be returned to the caller.
const supplier = new MySupplier("acetone", 5); // Limit to 5 results
for await (const product of supplier) {
// Will yield at most 5 products
}
protected limit: number;
ProtectedproductsThe products that are currently being built by the supplier. This array holds ProductBuilder instances that are in the process of being transformed into complete Product objects.
await supplier.queryProducts("acetone");
console.log(`Building ${supplier.products.length} products`);
for (const builder of supplier.products) {
const product = await builder.build();
console.log("Built product:", product.title);
}
protected products: ProductBuilder<T>[] = [];
ProtectedrequestCounter for HTTP requests made during the current query execution. This is used to track the number of requests and ensure we don't exceed the httpRequestHardLimit.
0
await supplier.queryProducts("acetone");
console.log(`Made ${supplier.requestCount} requests`);
if (supplier.requestCount >= supplier.httpRequestHardLimit) {
console.log("Reached request limit");
}
protected requestCount: number = 0;
ProtectedminMinimum number of milliseconds between two consecutive tasks
protected minConcurrentCycle: number = 100;
ProtectedheadersHTTP headers used as a basis for all requests to the supplier. These headers are merged with any request-specific headers when making HTTP requests.
class MySupplier extends SupplierBase<Product> {
constructor() {
super();
this.headers = {
"Accept": "application/json",
"User-Agent": "ChemPal/1.0"
};
}
}
protected headers: HeadersInit = {};
Protected ReadonlychallengeNumber of times fetch retries a request that comes back 403. Some
suppliers sit behind a WAF that 403s the first hit while planting a
session cookie (a "cookie handshake"); because every request now sets
credentials: "include", that cookie lands in the jar and the retry
carries it back, usually passing. We can't gate on the Set-Cookie
header (it's fetch-forbidden and invisible to JS), so this per-supplier
flag is the gate — 0 (the default) means never retry. Only enable it
for suppliers known to do this handshake.
0
protected readonly challengeRetryLimit: number = 0;
Protected ReadonlychallengeDelay in milliseconds between 403 challenge retries. Gives the WAF a
brief beat before re-requesting with the freshly-planted cookie.
300
protected readonly challengeRetryDelayMs: number = 300;
ProtectedloggerLogger for the supplier. Initialized in the constructor with the name of the inheriting class.
ProtectedproductDefault values for products. These will get overridden if they're found in the product data.
ProtectedcacheCache instance for this supplier.
Initialized after construction by initCache() (called from
SupplierFactory once supplierName is set). The ! assertion is safe
here because every code path that reads this.cache
(queryProductsWithCache, getProductData, getProductDataWithCache)
runs only after execute() is called on a factory-built instance, and the
factory always calls initCache() before handing the instance out.
ProtectedexcludedProduct-data cache keys the user has explicitly excluded via the "Ignore
Product" context menu action. Loaded once per execute() from
storage.local so membership checks are synchronous on the hot path
(see getProductData). Newly-ignored products take effect on the next
search, which matches the stated feature requirement.
ReadonlysupplierThe name of the supplier (used for display name, lists, etc).
ReadonlybaseThe base URL for the supplier.
ReadonlyshippingThe shipping scope of the supplier. Used to determine the shipping scope of the supplier.
ReadonlycountryThe country code of the supplier. Used to determine the currency and other country-specific information.
ReadonlypaymentThe payment methods accepted by the supplier. Used to determine the payment methods accepted by the supplier.
Protected ReadonlyfuzzFuzz scorer used by fuzzyFilter to score each candidate's title against
the query. Any function from fuzzball with the
(str1, str2, opts?) => number shape works. Subclasses override this when
a supplier's title format needs a different scorer (e.g. a catalog that
pads titles with boilerplate might prefer partial_ratio). Defaults to
ratio.
Overridable at runtime from userSettings.fuzzScorerOverride — see
setFuzzScorerOverride and fuzzyFilter below. The user's Advanced
settings selection wins over this subclass default when set.
Protected ReadonlyminThe minimum match percentage for a product to be considered a match.
ProtectedqueryIf the products first require a query of a search page that gets iterated over, those results are stored here. Acts as a cache for the initial search results before they are processed into full product objects.
ProtectedhttpMaximum number of HTTP requests allowed per search query. This is a hard limit to prevent excessive requests to the supplier's API. If this limit is reached, the supplier will stop making new requests.
50
class MySupplier extends SupplierBase<Product> {
constructor() {
super();
this.httpRequestHardLimit = 100; // Allow more requests
}
}
protected httpRequestHardLimit: number = 150;
ProtectedhttpProtectedmaxNumber of requests to process in parallel when fetching product details. This controls the batch size for concurrent requests to avoid overwhelming the supplier's API and the user's bandwidth.
10
class MySupplier extends SupplierBase<Product> {
constructor() {
super();
// Process 5 requests at a time
this.maxConcurrentRequests = 5;
}
}
protected maxConcurrentRequests: number = 5;
Protected ReadonlyrequiredCookies that must be written into the browser jar before any request runs
— e.g. a currency or session-preference cookie the backend reads. Seeded
once per instance by ensureSetup (before setup) via chrome.cookies,
since the Cookie request header is on the fetch-forbidden list and can't
be set through this.headers. Each entry's url defaults to baseURL.
Subclasses override this instead of hand-rolling a setup that calls
chrome.cookies.set directly.
[]
class MySupplier extends SupplierBase<Partial<Product>, Product> {
protected readonly requiredCookies: SupplierCookieSeed[] = [
{ name: "currency", value: "2" },
];
}
protected readonly requiredCookies: SupplierCookieSeed[] = [{ name: "currency", value: "2" }];
Supplier implementation for S3 Chemicals, a German based chemical supplier (shop.es-drei.de) built on the Shopware 5 platform.
Price currency is not hardcoded — the storefront serves either EUR or USD depending on session detection, so the currency is always read from the
meta[itemprop="priceCurrency"]tag on the detail page (with the visible symbol inferred from the rendered price text).Type Param: S
The supplier-specific product type (Partial)
Type Param: T
The common Product type that all suppliers map to
Example
Source