CIRCL hashlookup

CIRCL hashlookup (hashlookup.circl.lu)

CIRCL hash lookup is a public API to lookup hash values against known database of files. NSRL RDS database is included. More database will be included in the future. The API is accessible via HTTP ReST API and the API is also described as an OpenAPI.

Is it a database of malicious or non-malicious hash of files?

CIRCL hashlookup service only gives details about known files appearing in specific database(s). This gives you context and information about file hashes which can be discovered during investigation or digital forensic analysis.

API Usage

Get information about the hash lookup database (via ReST)

curl -X 'GET' \
  'https://hashlookup.circl.lu/info' \
  -H 'accept: application/json'
 1{
 2  "nsrl-version": "RDS Verion 2.73.1 - July 2021",
 3  "nsrl-NSRL-items": "165968856",
 4  "nsrl-NSRL-Legacy-items": "113737918",
 5  "nsrl-Android-items": "33419323",
 6  "nsrl-iOS-items": "46447082",
 7  "nsrl-NSRLMfg": "92353",
 8  "nsrl-NSRLOS": "1331",
 9  "nsrl-NSRLProd": "19050",
10  "hashlookup-version": "1.0"
11}

Perform an MD5 hash lookup

curl -X 'GET' \
  'https://hashlookup.circl.lu/lookup/md5/8ED4B4ED952526D89899E723F3488DE4' \
  -H 'accept: application/json'
 1{
 2  "CRC32": "7A5407CA",
 3  "FileName": "wow64_microsoft-windows-i..timezones.resources_31bf3856ad364e35_10.0.16299.579_de-de_f24979c73226184d.manifest",
 4  "FileSize": "2520",
 5  "MD5": "8ED4B4ED952526D89899E723F3488DE4",
 6  "OpSystemCode": {
 7    "MfgCode": "1006",
 8    "OpSystemCode": "362",
 9    "OpSystemName": "TBD",
10    "OpSystemVersion": "none"
11  },
12  "ProductCode": {
13    "ApplicationType": "Security",
14    "Language": "Multilanguage",
15    "MfgCode": "608",
16    "OpSystemCode": "868",
17    "ProductCode": "190742",
18    "ProductName": "Cumulative Update for Windows Server 2016 for x64 (KB4338817)",
19    "ProductVersion": "1709"
20  },
21  "SHA-1": "00000079FD7AAC9B2F9C988C50750E1F50B27EB5",
22  "SpecialCode": ""
23}

Perform an SHA-1 hash lookup

curl -X 'GET'   'https://hashlookup.circl.lu/lookup/sha1/FFFFFDAC1B1B4C513896C805C2C698D9688BE69F'   -H 'accept: application/json' | jq .
 1{
 2  "CRC32": "CBD64CD9",
 3  "FileName": ".rela.dyn",
 4  "FileSize": "240",
 5  "MD5": "131312A96CAD4ACAA7E2631A34A0D47C",
 6  "OpSystemCode": {
 7    "MfgCode": "1006",
 8    "OpSystemCode": "362",
 9    "OpSystemName": "TBD",
10    "OpSystemVersion": "none"
11  },
12  "ProductCode": {
13    "ApplicationType": "Operating System",
14    "Language": "English",
15    "MfgCode": "1722",
16    "OpSystemCode": "599",
17    "ProductCode": "163709",
18    "ProductName": "BlackArch Linux",
19    "ProductVersion": "2017.03.01"
20  },
21  "SHA-1": "FFFFFDAC1B1B4C513896C805C2C698D9688BE69F",
22  "SpecialCode": ""
23}

Bulk search of MD5 hashes

curl -X 'POST'   'https://hashlookup.circl.lu/bulk/md5' -H "Content-Type: application/json"  -d "{\"hashes\": [\"6E2F8616A01725DCB37BED0A2495AEB2\", \"8ED4B4ED952526D89899E723F3488DE4\", \"344428FA4BA313712E4CA9B16D089AC4\"]}" | jq .
 1[
 2  {
 3    "CRC32": "E774FD92",
 4    "FileName": "network",
 5    "FileSize": "7279",
 6    "MD5": "6E2F8616A01725DCB37BED0A2495AEB2",
 7    "OpSystemCode": "362",
 8    "ProductCode": "8321",
 9    "SHA-1": "00000903319A8CE18A03DFA22C07C6CA43602061",
10    "SpecialCode": ""
11  },
12  {
13    "CRC32": "7A5407CA",
14    "FileName": "wow64_microsoft-windows-i..timezones.resources_31bf3856ad364e35_10.0.16299.579_de-de_f24979c73226184d.manifest",
15    "FileSize": "2520",
16    "MD5": "8ED4B4ED952526D89899E723F3488DE4",
17    "OpSystemCode": "362",
18    "ProductCode": "190742",
19    "SHA-1": "00000079FD7AAC9B2F9C988C50750E1F50B27EB5",
20    "SpecialCode": ""
21  },
22  {
23    "CRC32": "7516A25F",
24    "FileName": ".text._ZNSt14overflow_errorC1ERKSs",
25    "FileSize": "33",
26    "MD5": "344428FA4BA313712E4CA9B16D089AC4",
27    "OpSystemCode": "362",
28    "ProductCode": "219181",
29    "SHA-1": "0000001FFEF4BE312BAB534ECA7AEAA3E4684D85",
30    "SpecialCode": ""
31  }
32]

Bulk search of SHA-1 hashes

curl -X 'POST'   'https://hashlookup.circl.lu/bulk/sha1' -H "Content-Type: application/json"  -d "{\"hashes\": [\"FFFFFDAC1B1B4C513896C805C2C698D9688BE69F\", \"FFFFFF4DB8282D002893A9BAF00E9E9D4BA45E65\", \"FFFFFE4C92E3F7282C7502F1734B243FA52326FB\"]}" | jq .
 1[
 2  {
 3    "CRC32": "CBD64CD9",
 4    "FileName": ".rela.dyn",
 5    "FileSize": "240",
 6    "MD5": "131312A96CAD4ACAA7E2631A34A0D47C",
 7    "OpSystemCode": "362",
 8    "ProductCode": "163709",
 9    "SHA-1": "FFFFFDAC1B1B4C513896C805C2C698D9688BE69F",
10    "SpecialCode": ""
11  },
12  {
13    "CRC32": "8654F11A",
14    "FileName": "s_copypix.c",
15    "FileSize": "19541",
16    "MD5": "559D049F44942683093A91BA19D0AF54",
17    "OpSystemCode": "362",
18    "ProductCode": "215139",
19    "SHA-1": "FFFFFF4DB8282D002893A9BAF00E9E9D4BA45E65",
20    "SpecialCode": ""
21  },
22  {
23    "CRC32": "8E51A269",
24    "FileName": "358.git2-msvstfs.dll",
25    "FileSize": "65",
26    "MD5": "9E4C165089CBA3653484C3F23F1CBC67",
27    "OpSystemCode": "362",
28    "ProductCode": "201317",
29    "SHA-1": "FFFFFE4C92E3F7282C7502F1734B243FA52326FB",
30    "SpecialCode": ""
31  }
32]

API and HTTP return codes

HTTP return code Description and Interpretation
200 200 means the searched hash is present in at least one of the database
404 404 means the searched hash is not present in the any of the database
400 400 means the input used for the hash is in an incorrect format

Querying the hashlookup database via DNS

The domain to query is <query>.dns.hashlookup.circl.lu. The query can be info or an MD5 or SHA-1 value.

Info of the hashlookup database

dig +short -t TXT info.dns.hashlookup.circl.lu | jq -r . | jq .

 1{
 2  "nsrl-version": "RDS Verion 2.73.1 - July 2021",
 3  "nsrl-NSRL-items": "165968856",
 4  "nsrl-Android-items": "33419323",
 5  "nsrl-iOS-items": "46447082",
 6  "nsrl-NSRLMfg": "543004",
 7  "nsrl-NSRLOS": "6414",
 8  "nsrl-NSRLProd": "333546",
 9  "hashlookup-version": "0.1"
10}

Query of a hash

dig +short -t TXT 931606baaa7a2b4ef61198406f8fc3f4.dns.hashlookup.circl.lu | jq -r . | jq .

 1{
 2  "CRC32": "13C49389",
 3  "FileName": "ls",
 4  "FileSize": "133792",
 5  "MD5": "931606BAAA7A2B4EF61198406F8FC3F4",
 6  "OpSystemCode": "362",
 7  "ProductCode": "217853",
 8  "SHA-1": "D3A21675A8F19518D8B8F3CEF0F6A21DE1DA6CC7",
 9  "SpecialCode": ""
10}

Sample use-cases

How to quickly check a set of files in a local directory?

sha1sum * | cut -f1 -d" " | parallel 'curl -s https://hashlookup.circl.lu/lookup/sha1/{}' | jq .

Negative output (hash not existing in the database) can be excluded with the -f option of curl.

sha1sum * | cut -f1 -d" " | parallel 'curl -f -s https://hashlookup.circl.lu/lookup/sha1/{}' | jq .

Libraries and Software available to use CIRCL hashlookup