ChemPare Documentation - v0.0.0
    Preparing search index...

    Class SupplierLaboratoriumDiscounter

    Class for retrieving search results and iterating over Laboratorium Discounter online web store.

    Laboratorium Discounters seems to use Lightspeed eCom (webshopapp) as their ecommerce platform, as can be determined by loking at the shop.domains.main value of a search response, or looking at where some of their assets are pulled from (cdn.webshopapp.com).

    Laboratorium Discounters API is pretty easy to use, and the search results are in JSON format. It looks like any page (including home page) can be displayed in JSON format if you append ?format=json to the URL.

    But to get the variants or other product specific data, you need to fetch the product details page.

    • Nitric acid (JSON)
      • With all the product specific data found at product and variants at product.variants.

    Links:

    > [!IMPORTANT] > Be careful that your scripts do not produce too many XHR calls. A few (2-3) calls per page or making > calls based on user input could be acceptable, but letting users do multiple calls in a short period of time > could see them BANNED from shops. Please only use these methods as workarounds in specific instances.

    const supplier = new SupplierLaboratoriumDiscounter();
    for await (const product of supplier) {
      console.log(product);
    }

    Hierarchy (View Summary)

    Implements

    Index

    Constructors

    constructor

    Methods

    initCache setup httpGetHeaders httpPost httpPostJson httpPostHtml httpGet fuzzyFilter httpGetHtml httpGetJson queryProductsWithCache execute finishProduct href getProductDataWithCache groupVariants fetch makeQueryParams queryProducts titleSelector initProductBuilders getProductData

    Properties

    query baseSearchParams controller limit products httpRequestHardLimit requestCount maxConcurrentRequests minConcurrentCycle logger productDefaults productDataCacheKey cache supplierName baseURL shipping country paymentMethods queryResults httpRequstCount headers categoryIds

    Constructors

    • Creates a new instance of the supplier base class. Initializes the supplier with query parameters, request limits, and abort controller. Sets up logging and default product values.

      Parameters

      • query: string

        The search term to query products for

      • limit: number = 5

        The maximum number of results to return (default: 5)

      • Optionalcontroller: AbortController

        AbortController instance for managing request cancellation

      Returns SupplierLaboratoriumDiscounter

      // Create a supplier with default limit
      const supplier = new MySupplier("sodium chloride", undefined, new AbortController());

      // Create a supplier with custom limit
      const supplier = new MySupplier("acetone", 10, new AbortController());

      // Create a supplier and handle cancellation
      const controller = new AbortController();
      const supplier = new MySupplier("ethanol", 5, controller);

      // Later, to cancel all pending requests:
      controller.abort();

    Methods

    • Initializes the cache for the supplier. This is called after construction to ensure supplierName is set.

      Returns void

      The cache is initialized with the supplier's name and is used to store both query results and product data. This method should be called after the supplier's name is set to ensure proper cache key generation.

      class MySupplier extends SupplierBase<Product> {
        constructor() {
          super("acetone", 5);
          // supplierName is set here
          this.initCache(); // Initialize cache after supplierName is set
        }
      }

    • Placeholder for any setup that needs to be done before the query is made. Override this in subclasses if you need to perform setup (e.g., authentication, token fetching).

      Returns Promise<void>

      A promise that resolves when the setup is complete.

      await supplier.setup();

    • Retrieves HTTP headers from a URL using a HEAD request. Useful for checking content types, caching headers, and other metadata without downloading the full response.

      Parameters

      • url: string | URL

        The URL to fetch headers from

      Returns Promise<Maybe<HeadersInit>>

      Promise resolving to the response headers or void if request fails

      // Basic usage
      const headers = await supplier.httpGetHeaders('https://example.com/product/123');
      if (headers) {
        console.log('Content-Type:', headers['content-type']);
      }

      // With error handling
      try {
        const headers = await supplier.httpGetHeaders('https://example.com/product/123');
        if (headers) {
          console.log('Headers:', headers);
        }
      } catch (err) {
        console.error('Failed to fetch headers:', err);
      }

    • Sends a POST request to the given URL with the given body and headers. Handles request setup, error handling, and response caching.

      Parameters

      Returns Promise<Maybe<Response>>

      Promise resolving to the Response object or void if request fails

      // Basic POST request
      const response = await supplier.httpPost({
        path: '/api/v1/products',
        body: { name: 'Test Chemical' }
      });

      // POST with custom headers
      const response = await supplier.httpPost({
        path: '/api/v1/products',
        body: { name: 'Test Chemical' },
        headers: {
          'Authorization': 'Bearer token123',
          'Content-Type': 'application/json'
        }
      });

      // POST with custom host and params
      const response = await supplier.httpPost({
        path: '/api/v1/products',
        host: 'api.example.com',
        body: { name: 'Test Chemical' },
        params: { version: '2' }
      });

      // Error handling
      try {
        const response = await supplier.httpPost({ path: '/api/v1/products', body: { name: 'Test' } });
        if (response && response.ok) {
          const data = await response.json();
          console.log('Created:', data);
        }
      } catch (err) {
        console.error('POST failed:', err);
      }

    • Sends a POST request and returns the response as a JSON object.

      Parameters

      Returns Promise<Maybe<JsonValue>>

      The response from the POST request as a JSON object.

      // Basic usage
      const data = await supplier.httpPostJson({
        path: '/api/v1/products',
        body: { name: 'John' }
      });

      // With custom headers and error handling
      try {
        const data = await supplier.httpPostJson({
          path: '/api/v1/products',
          body: { name: 'John' },
          headers: { 'Authorization': 'Bearer token123' }
        });
        if (data) {
          console.log('Created:', data);
        }
      } catch (err) {
        console.error('POST JSON failed:', err);
      }

    • Sends a POST request and returns the response as a HTML string.

      Parameters

      Returns Promise<Maybe<string>>

      Promise resolving to the HTML response as a string or void if request fails

      TypeError - If the response is not valid HTML content

      // Basic usage
      const html = await supplier.httpPostHtml({
        path: '/api/v1/products',
        body: { name: 'John' }
      });

    • Sends a GET request to the given URL with the specified options. Handles request setup, error handling, and response caching.

      Parameters

      Returns Promise<Maybe<Response>>

      Promise resolving to the Response object or void if request fails

      // Basic GET request
      const response = await supplier.httpGet({
        path: '/products/search',
        params: { query: 'sodium chloride' }
      });

      // GET with custom headers
      const response = await supplier.httpGet({
        path: '/products/search',
        headers: { 'Accept': 'application/json' }
      });

      // GET with custom host
      const response = await supplier.httpGet({
        path: '/products/search',
        host: 'api.example.com',
        params: { category: 'chemicals' }
      });

      // Error handling
      try {
        const response = await supplier.httpGet({ path: '/products/search' });
        if (response && response.ok) {
          const data = await response.json();
          console.log('Products:', data);
        }
      } catch (err) {
        console.error('GET failed:', err);
      }

    • Filters an array of data using fuzzy string matching to find items that closely match a query string. Uses the WRatio algorithm from fuzzball for string similarity comparison.

      Type Parameters

      • X

      Parameters

      • query: string

        The search string to match against

      • data: X[]

        Array of data objects to search through

      • cutoff: number = 40

        Minimum similarity score (0-100) for a match to be included (default: 40)

      Returns X[]

      Array of matching data objects with added fuzzy match metadata

      // Example with simple string array
      const products = [
        { title: "Sodium Chloride", price: 29.99 },
        { title: "Sodium Hydroxide", price: 39.99 },
        { title: "Potassium Chloride", price: 19.99 }
      ];

      const matches = this.fuzzyFilter("sodium chloride", products);
      // Returns: [
      //   {
      //     title: "Sodium Chloride",
      //     price: 29.99,
      //     ___fuzz: { score: 100, idx: 0 }
      //   },
      //   {
      //     title: "Sodium Hydroxide",
      //     price: 39.99,
      //     ___fuzz: { score: 85, idx: 1 }
      //   }
      // ]

      // Example with custom cutoff
      const strictMatches = this.fuzzyFilter("sodium chloride", products, 90);
      // Returns only exact matches with score >= 90

      // Example with different data structure
      const chemicals = [
        { name: "NaCl", formula: "Sodium Chloride" },
        { name: "NaOH", formula: "Sodium Hydroxide" }
      ];

      // Override titleSelector to use formula field
      this.titleSelector = (data) => data.formula;
      const formulaMatches = this.fuzzyFilter("sodium chloride", chemicals);

    • Makes an HTTP GET request and returns the response as a string. Handles request configuration, error handling, and HTML parsing.

      Parameters

      Returns Promise<Maybe<string>>

      Promise resolving to the HTML response as a string or void if request fails

      TypeError - If the response is not valid HTML content

      // Basic GET request
      const html = await this.httpGetHtml({
        path: "/api/products",
        params: { search: "sodium" }
      });

      // GET request with custom headers
      const html = await this.httpGetHtml({
        path: "/api/products",
        headers: {
          "Authorization": "Bearer token123",
          "Accept": "text/html"
        }
      });

      // GET request with custom host
      const html = await this.httpGetHtml({
        path: "/products",
        host: "api.supplier.com",
        params: { limit: 10 }
      });

    • Makes an HTTP GET request and returns the response as parsed JSON. Handles request configuration, error handling, and JSON parsing.

      Parameters

      Returns Promise<Maybe<JsonValue>>

      Promise resolving to the parsed JSON response or void if request fails

      TypeError - If the response is not valid JSON content

      // Basic GET request
      const data = await supplier.httpGetJson({ path: '/api/products', params: { search: 'sodium' } });

      // GET request with custom headers
      const data = await supplier.httpGetJson({
        path: '/api/products',
        headers: {
          'Authorization': 'Bearer token123',
          'Accept': 'application/json'
        }
      });

      // GET request with custom host
      const data = await supplier.httpGetJson({
        path: '/products',
        host: 'api.supplier.com',
        params: { limit: 10 }
      });

      // Error handling
      try {
        const data = await supplier.httpGetJson({ path: '/api/products' });
        if (data) {
          console.log('Products:', data);
        }
      } catch (error) {
        console.error('Failed to fetch products:', error);
      }

    • Executes a product search query with caching support. First checks the cache for existing results, then falls back to the actual query if needed. The limit parameter is only used for the actual query and doesn't affect caching.

      Parameters

      • query: string

        The search term to query products for

      • limit: number = ...

        The maximum number of results to return (defaults to instance limit)

      Returns Promise<void | ProductBuilder<Product>[]>

      Promise resolving to array of product builders or void if search fails

      // Basic usage with default limit
      const results = await supplier.queryProductsWithCache("acetone");
      if (results) {
        console.log(`Found ${results.length} products`);
      }

      // With custom limit
      const results = await supplier.queryProductsWithCache("acetone", 10);
      if (results) {
        for (const builder of results) {
          const product = await builder.build();
          console.log(product.title, product.price);
        }
      }

    • Executes the supplier's search query and returns the results. This method will execute all results concurrently (to the limits set in the supplier class), and resolve to an array of product objects.

      Returns AsyncGenerator<Product, void, undefined>

      Promise resolving to an array of products

      This method is used to execute the supplier's search query and return the results.

    • Finalizes a partial product by adding computed properties and validating the result. This method:

      1. Validates the product has minimal required properties
      2. Computes USD price if product is in different currency
      3. Calculates base quantity using the unit of measure
      4. Ensures the product URL is absolute

      Parameters

      Returns Promise<Maybe<Product>>

      Promise resolving to a complete Product object or void if validation fails

      // Example with a valid partial product
      const builder = new ProductBuilder<Product>(this.baseURL);
      builder
        .setBasicInfo("Sodium Chloride", "/products/nacl", "ChemSupplier")
        .setPricing(29.99, "USD", "$")
        .setQuantity(500, "g");

      const finishedProduct = await this.finishProduct(builder);
      if (finishedProduct) {
        console.log("Finalized product:", {
          title: finishedProduct.title,
          price: finishedProduct.price,
          quantity: finishedProduct.quantity,
          uom: finishedProduct.uom,
          usdPrice: finishedProduct.usdPrice,
          baseQuantity: finishedProduct.baseQuantity
        });
      }

      // Example with an invalid partial product
      const invalidBuilder = new ProductBuilder<Product>(this.baseURL);
      invalidBuilder.setBasicInfo("Sodium Chloride", "/products/nacl", "ChemSupplier");
      // Missing required fields

      const invalidProduct = await this.finishProduct(invalidBuilder);
      if (!invalidProduct) {
        console.log("Failed to finalize product - missing required fields");
      }

    • Takes in either a relative or absolute URL and returns an absolute URL. This is useful for when you aren't sure if the link (retrieved from parsed text, a setting, an element, an anchor value, etc) is absolute or not. Using relative links will result in http://chrome-extension://... being added to the link.

      Parameters

      • path: string | URL

        URL object or string

      • Optionalparams: Maybe<RequestParams>

        The parameters to add to the URL.

      • Optionalhost: string

        The host to use for overrides (eg: needing to call a different host for an API)

      Returns string

      absolute URL

      this.href('/some/path')
      // https://supplier_base_url.com/some/path

      this.href('https://supplier_base_url.com/some/path', null, 'another_host.com')
      // https://another_host.com/some/path

      this.href('/some/path', { a: 'b', c: 'd' }, 'another_host.com')
      // http://another_host.com/some/path?a=b&c=d

      this.href('https://supplier_base_url.com/some/path')
      // https://supplier_base_url.com/some/path

      this.href(new URL('https://supplier_base_url.com/some/path'))
      // https://supplier_base_url.com/some/path

      this.href('/some/path', { a: 'b', c: 'd' })
      // https://supplier_base_url.com/some/path?a=b&c=d

      this.href('https://supplier_base_url.com/some/path', new URLSearchParams({ a: 'b', c: 'd' }))
      // https://supplier_base_url.com/some/path?a=b&c=d

    • Retrieves product data with caching support. Similar to getProductData but allows for additional parameters to be included in the cache key.

      Parameters

      Returns Promise<void | ProductBuilder<Product>>

      Promise resolving to the updated ProductBuilder or void if fetch fails

      const builder = new ProductBuilder<Product>(this.baseURL);
      builder.setBasicInfo("Acetone", "/products/acetone", "ChemSupplier");

      // Use custom fetcher with additional params
      const updatedBuilder = await supplier.getProductDataWithCache(
        builder,
        async (b) => {
          // Custom fetching logic
          return b;
        },
        { version: "2.0" }
      );

    • Groups variants of a product by their title

      Type Parameters

      • R

      Parameters

      • data: R[]

        Array of product listings from search results

      Returns R[]

      Array of product listings with grouped variants

      Create a generic method for this, the same method is used in Synthetika and could be of use with LoudWolf.

      const results = await this.queryProducts("sodium chloride");
      const grouped = this.groupVariants(results);
      // grouped is an array of product listings with grouped variants

    • Internal fetch method with request counting and decorator. Tracks request count and enforces hard limits on HTTP requests.

      Parameters

      • ...args: [input: URL | RequestInfo, init?: RequestInit]

        Arguments to pass to fetchDecorator (usually a Request or URL and options)

      Returns Promise<any>

      The response from the fetchDecorator

      Error if request count exceeds hard limit

      // Example usage inside a subclass:
      const response = await this.fetch(new Request('https://example.com'));
      if (response.ok) {
        const data = await response.json();
        console.log(data);
      }

      // With custom request options
      const response = await this.fetch(
        new Request('https://example.com', {
          headers: { 'Accept': 'application/json' }
        })
      );

    • Constructs the query parameters for a product search request

      Parameters

      • Optionallimit: number

        The maximum number of results to query for

      Returns LaboratoriumDiscounterSearchParams

      Object containing all required search parameters

      const params = this.makeQueryParams(20);
      // Returns: { limit: "20", format: "json" }

      // Use in search request
      const response = await this.httpGetJson({
        path: "/en/search/chemical",
        params: this.makeQueryParams(20)
      });

    • Executes a product search query and returns matching products

      Parameters

      • query: string

        Search term to look for

      • limit: number = ...

        The maximum number of results to query for

      Returns Promise<void | ProductBuilder<Product>[]>

      Promise resolving to array of product objects or void if search fails

      // Search for sodium chloride with a limit of 10 results
      const products = await this.queryProducts("sodium chloride", 10);
      if (products) {
        console.log(`Found ${products.length} products`);
        for (const product of products) {
          const builtProduct = await product.build();
          console.log(builtProduct.title, builtProduct.price);
        }
      } else {
        console.error("No products found or search failed");
      }

    • Initialize product builders from Laboratorium Discounter search response data. Transforms product listings into ProductBuilder instances, handling:

      • Basic product information (title, URL, supplier)
      • Product descriptions and content
      • Product IDs and SKUs
      • Availability status
      • CAS number extraction from product content
      • Quantity parsing from variant information
      • Product codes and EANs

      Parameters

      Returns ProductBuilder<Product>[]

      Array of ProductBuilder instances initialized with product data

      const results = await this.queryProducts("sodium chloride");
      if (results) {
        const builders = this.initProductBuilders(results);
        // Each builder contains parsed product data
        for (const builder of builders) {
          const product = await builder.build();
          console.log({
            title: product.title,
            price: product.price,
            quantity: product.quantity,
            uom: product.uom,
            cas: product.cas
          });
        }
      }

    Properties

    query: string

    String to query for (Product name, CAS, etc). This is the search term that will be used to find products. Set during construction and used throughout the supplier's lifecycle.

    const supplier = new MySupplier("sodium chloride", 10);
    console.log(supplier.query); // "sodium chloride"
    baseSearchParams: Record<string, string | number> = {}

    The base search parameters that are always included in search requests. These parameters are merged with any additional search parameters when making requests to the supplier's API.

    class MySupplier extends SupplierBase<Product> {
      constructor() {
        super();
        this.baseSearchParams = {
          format: "json",
          version: "2.0"
        };
      }
    }
    controller: AbortController

    The AbortController instance used to manage and cancel ongoing requests. This allows for cancellation of in-flight requests when needed, such as when a new search is started or the supplier is disposed.

    const controller = new AbortController();
    const supplier = new MySupplier("acetone", 5, controller);

    // Later, to cancel all pending requests:
    controller.abort();
    limit: number

    The maximum number of results to return for a search query. This is not a limit on HTTP requests, but rather the number of products that will be returned to the caller.

    const supplier = new MySupplier("acetone", 5); // Limit to 5 results
    for await (const product of supplier) {
      // Will yield at most 5 products
    }
    products: ProductBuilder<Product>[] = []

    The products that are currently being built by the supplier. This array holds ProductBuilder instances that are in the process of being transformed into complete Product objects.

    await supplier.queryProducts("acetone");
    console.log(`Building ${supplier.products.length} products`);
    for (const builder of supplier.products) {
      const product = await builder.build();
      console.log("Built product:", product.title);
    }
    httpRequestHardLimit: number = 50

    Maximum number of HTTP requests allowed per search query. This is a hard limit to prevent excessive requests to the supplier's API. If this limit is reached, the supplier will stop making new requests.

    50

    class MySupplier extends SupplierBase<Product> {
      constructor() {
        super();
        this.httpRequestHardLimit = 100; // Allow more requests
      }
    }
    requestCount: number = 0

    Counter for HTTP requests made during the current query execution. This is used to track the number of requests and ensure we don't exceed the httpRequestHardLimit.

    0

    await supplier.queryProducts("acetone");
    console.log(`Made ${supplier.requestCount} requests`);
    if (supplier.requestCount >= supplier.httpRequestHardLimit) {
      console.log("Reached request limit");
    }
    maxConcurrentRequests: number = 3

    Number of requests to process in parallel when fetching product details. This controls the batch size for concurrent requests to avoid overwhelming the supplier's API and the user's bandwidth.

    10

    class MySupplier extends SupplierBase<Product> {
      constructor() {
        super();
        // Process 5 requests at a time
        this.maxConcurrentRequests = 5;
      }
    }
    minConcurrentCycle: number = 100

    Minimum number of milliseconds between two consecutive tasks

    logger: Logger
    productDefaults: {
        uom: string;
        quantity: number;
        currencyCode: string;
        currencySymbol: string;
    } = ...
    productDataCacheKey: "supplier_product_data_cache"
    cache: SupplierCache
    supplierName: string = "Laboratorium Discounter"

    The name of the supplier

    baseURL: string = "https://www.laboratoriumdiscounter.nl"

    The base URL for the supplier

    shipping: ShippingRange = "domestic"

    The shipping scope of the supplier. This is used to determine the shipping scope of the supplier.

    country: string = "NL"

    The country code of the supplier. This is used to determine the currency and other country-specific information.

    paymentMethods: PaymentMethod[] = ...

    The payment methods accepted by the supplier. This is used to determine the payment methods accepted by the supplier.

    queryResults: LaboratoriumDiscounterProductObject[] = []

    If the products first require a query of a search page that gets iterated over, those results are stored here. This acts as a cache for the initial search results before they are processed into full product objects.

    // After a search query
    await supplier.queryProducts("acetone");
    console.log(`Found ${supplier.queryResults.length} initial results`);
    httpRequstCount: number = 0
    headers: HeadersInit = ...

    HTTP headers used as a basis for all requests to the supplier. These headers are merged with any request-specific headers when making HTTP requests.

    class MySupplier extends SupplierBase<Product> {
      constructor() {
        super();
        this.headers = {
          "Accept": "application/json",
          "User-Agent": "ChemCrawler/1.0"
        };
      }
    }
    categoryIds: number[] = ...

    Category IDs to include in the search (and including any that aren't in these categories or their sub-categories).

    Settings

    Member Visibility

    On This Page

    Constructors
    Methods
    Properties