InfoQuick links for Builders

Node Versions and Upgrades

You can find a list of release notes for Chainlink nodes in the smartcontractkit GitHub repository. Docker images are available in the Chainlink Docker hub.

Changes in v2.13.0 nodes - 2024-07-01

v2.13.0 release notes

Breaking Change

  • Removed the xdai ChainType config option. Moving forward, only gnosis can be used.

Added

  • Added a mechanism to validate forwarders for OCR2 and fallback to EOA if necessary
  • Added an auto-purge feature to the EVM TXM that identifies terminally stuck transactions either through a chain specific method or heurisitic then purges them to unblock the nonce. Included four new toml configs under Transactions.AutoPurge to configure this new feature: Enabled, Threshold, MinAttempts, and DetectionApiUrl.
  • Add option to include GasPriceSubunits pipeline to include gasPriceSubunits in median ocr2 transmission. Use this only with the Starknet chain for now.
  • Added config option HeadTracker.FinalityTagBypass to force HeadTracker to track blocks up to FinalityDepth even if FinalityTagsEnabled = true. This option is a temporary measure to address high CPU usage on chains with extremely large actual finality depth (gap between the current head and the latest finalized block).
  • Added config option HeadTracker.MaxAllowedFinalityDepth maximum gap between current head to the latest finalized block that HeadTracker considers healthy.

Updated

  • Fixed CPU usage issues caused by inefficiencies in HeadTracker. HeadTracker's support of finality tags caused a drastic increase in the number of tracked blocks on the Arbitrum chain (from 50 to 12,000), which led to a 30% increase in CPU usage. The fix improves the data structure for tracking blocks and makes lookup more efficient. BenchmarkHeadTracker_Backfill shows a 40x time reduction.
  • Fixed metric description on mercury_transmit_queue_load

DB Updated

  • Improve handling of Postgres connection settings and driver versions.

Changes in v2.12.0 nodes - 2024-06-05

v2.12.0 release notes

Added

  • #12867 27d9413286 Added a new CLI command, blocks find-lca, which finds the latest block that is available in both the database and on the chain for the specified chain. Added a new CLI command, node remove-blocks, which removes all blocks and logs greater than or equal to the specified block number. #nops #added
  • #12533 ccb8cd85fe Re-enable abandoned transaction tracker
  • #12760 3f4573479c Enable configurable client error regexes for error classification. New toml configuration options for [EVM.NodePool.Errors] to pass regexes on NonceTooLow, NonceTooHigh, ReplacementTransactionUnderpriced, LimitReached, TransactionAlreadyInMempool, TerminallyUnderpriced, InsufficientEth, TxFeeExceedsCap, L2FeeTooLow, L2FeeTooHigh, L2Full, TransactionAlreadyMined, Fatal, and ServiceUnavailable.
  • #12767 8db5ccfb39 Validate user email before asking for a password in the chainlink CLI.

Updated

  • #12605 1d9dd466e2 core/chains/evm/logpoller: Stricter finality checks in LogPoller, to be more robust during rpc failover events
  • #12595 e6d4814bda Move JuelsPerFeeCoinCacheDuration under JuelsPerFeeCoinCache struct in config. Rename JuelsPerFeeCoinCacheDuration to updateInterval. Add stalenessAlertThreshold to JuelsPerFeeCoinCache config. StalenessAlertThreshold cfg option has a default of 24 hours which means that it doesn't have to be set unless we want to override the duration after which a stale cache should start throwing errors.

Upcoming change in 2.13.0

  • #12093 3f6d901fe6 The xdai ChainType has been renamed to gnosis to match the chain's new name. The old value is still supported but has been deprecated and will be removed in v2.13.0.

Changes in v2.11.0 nodes - 2024-05-01

v2.11.0 release notes

Added

  • Added a tx simulation feature to the chain client to enable testing for zk out-of-counter (OOC) errors
  • Add the pool_rpc_node_highest_finalized_block metric that tracks the highest finalized block seen per RPC. If FinalityTagEnabled = true, a positive NodePool.FinalizedBlockPollInterval is needed to collect the metric. If the finality tag is not enabled, the metric is populated with a calculated latest finalized block based on the latest head and finality depth.

Updated

  • Moved JuelsPerFeeCoinCacheDuration under the JuelsPerFeeCoinCache struct in config. Rename JuelsPerFeeCoinCacheDuration to updateInterval. Add stalenessAlertThreshold to JuelsPerFeeCoinCache config. The StalenessAlertThreshold config option has a default of 24 hours which means that it doesn't have to be set unless we want to override the duration after which a stale cache should start throwing errors.
  • Updated config for Polygon zkEVM chains.
  • HeadTracker now respects the FinalityTagEnabled config option. If the flag is enabled, HeadTracker backfills blocks up to the latest finalized block provided by the corresponding RPC call. To address potential misconfigurations, HistoryDepth is now calculated from the latest finalized block instead of the head. Note that consumers like TXM and LogPoller do not fully use the Finality Tag yet.
  • Change LimitTransfer gasLimit type from uint32 to uint64

Upcoming changes in v2.13.0

  • The xdai ChainType has been renamed to gnosis to match the chain's new name. The old value is still supported but has been deprecated and will be removed in v2.13.0.

Changes in v2.10.0 nodes - 2024-04-08

v2.10.0 release notes

Added

  • Gas bumping logic to the SuggestedPriceEstimator. The bumping mechanism for this estimator refetches the price from the RPC and adds a buffer on top using the greater of BumpPercent and BumpMin.
  • Added a new configuration field named NodeIsSyncingEnabled for EVM.NodePool that will check on every reconnection to an RPC if it's syncing and should not be transitioned to Alive state. Disabled by default.
  • Add preliminary support for "llo" job type (Data Streams V1)
  • Add LogPrunePageSize parameter to the EVM configuration. This parameter controls the number of logs removed during prune phase in LogPoller. Default value is 0, which deletes all logs at once - exactly how it used to work, so it doesn't require any changes on the product's side.
  • Add Juels Fee Per Coin data source caching for OCR2 Feeds. Cache is time based and is turned on by default with default cache refresh of 5 minutes. Cache can be configured through pluginconfig using "juelsPerFeeCoinCacheDuration" and "juelsPerFeeCoinCacheDisabled" tags. Duration tag accepts values between "30s" and "20m" with default of "0s" that is overridden on cache startup to 5 minutes.
  • Add rebalancer support for feeds manager ocr2 plugins.

Fixed

  • P2P.V2 is required in configuration when either OCR or OCR2 are enabled. The node will fail to boot if P2P.V2 is not enabled.
  • Removed unnecessary gas price warnings in gas estimators when EIP-1559 mode is enabled.

Changed

  • Minimum required version of Postgres is now >= 12. Postgres 11 reached end-of-life (EOL) in November 2023. Added a new version check that will prevent Chainlink from running on EOL'd Postgres. If you are running Postgres <= 11 you must upgrade to the latest version. The check can be forcibly overridden by setting SKIP_PG_VERSION_CHECK=true.
  • Updated the LimitDefault and LimitMax configs types to uint64

Changes in v2.9.1 nodes - 2024-03-07

v2.9.1 release notes

Changed

  • eth_call RPC requests are now sent with both input and data fields to increase compatibility with servers that recognize only one.
  • GasEstimator will now include Type 0x3 (Blob) transactions in the gas calculations to estimate it more accurately.

Changes in v2.9.0 nodes - 2024-02-22

v2.9.0 release notes

Added

  • The chainlink health CLI command and HTML /health endpoint to provide human-readable views of the underlying JSON health data.

  • New job type stream to represent streamspecs. This job type is not yet used anywhere but will be required for Data Streams V1.

  • Environment variables CL_MEDIAN_ENV, CL_SOLANA_ENV, and CL_STARKNET_ENV for setting environment variables in LOOP Plugins with an .env file.

    echo "Foo=Bar" >> median.env
echo "Baz=Val" >> median.env
CL_MEDIAN_ENV="median.env"

Fixed

  • Fixed the encoding used for transactions when resending in batches.

Removed

  • P2P.V1 is no longer supported and must not be set in the TOML configuration in order to boot. Use P2P.V2 instead. If you are using both, V1 can simply be removed.

  • Removed TelemetryIngress.URL and TelemetryIngress.ServerPubKey from TOML configuration, these fields are replaced by [[TelemetryIngress.Endpoints]]:

      [[TelemetryIngress.Endpoints]]
  Network = '...' # e.g. EVM. Solana, Starknet, Cosmos
  ChainID = '...' # e.g. 1, 5, devnet, mainnet-beta
  URL = '...'
  ServerPubKey = '...'

Changes in v2.8.0 nodes - 2024-01-24

v2.8.0 release notes

Added

  • Added distributed tracing in the OpenTelemetry trace format to the node, currently focused at the LOOPP Plugin development effort. This includes a new set of Tracing TOML configurations. The default for collecting traces is off - you must explicitly enable traces and setup a valid OpenTelemetry collector. Refer to the README for more details.

  • Added a new, optional WebServer authentication option that supports LDAP as a user identity provider. This enables user login access and user roles to be managed and provisioned via a centralized remote server that supports the LDAP protocol, which can be helpful when running multiple nodes. See the documentation for more information and config setup instructions. There is a new [WebServer].AuthenticationMethod config option, when set to ldap requires the new [WebServer.LDAP] config section to be defined, see the reference docs/core.toml.

  • New prom metrics for mercury transmit queue: mercury_transmit_queue_delete_error_count mercury_transmit_queue_insert_error_count mercury_transmit_queue_push_error_count Nops should consider alerting on these.

  • Mercury now implements a local cache for fetching prices for fees, which ought to reduce latency and load on the mercury server, as well as increasing performance. It is enabled by default and can be configured with the following new config variables:

    [Mercury]

# Mercury.Cache controls settings for the price retrieval cache querying a mercury server
[Mercury.Cache]
# LatestReportTTL controls how "stale" we will allow a price to be e.g. if
# set to 1s, a new price will always be fetched if the last result was
# from 1 second ago or older.
#
# Another way of looking at it is such: the cache will _never_ return a
# price that was queried from now-LatestReportTTL or before.
#
# Setting to zero disables caching entirely.
LatestReportTTL = "1s" # Default
# MaxStaleAge is that maximum amount of time that a value can be stale
# before it is deleted from the cache (a form of garbage collection).
#
# This should generally be set to something much larger than
# LatestReportTTL. Setting to zero disables garbage collection.
MaxStaleAge = "1h" # Default
# LatestReportDeadline controls how long to wait for a response from the
# mercury server before retrying. Setting this to zero will wait indefinitely.
LatestReportDeadline = "5s" # Default

  • New prom metrics for the mercury cache: mercury_cache_fetch_failure_count mercury_cache_hit_count mercury_cache_wait_count mercury_cache_miss_count

  • Added new EVM.OCR TOML config fields DeltaCOverride and DeltaCJitterOverride for overriding the config DeltaC.

  • Mercury v0.2 has improved consensus around current block that uses the most recent 5 blocks instead of only the latest one

  • Two new prom metrics for mercury, nops should consider adding alerting on these:

    • mercury_insufficient_blocks_count
    • mercury_zero_blocks_count

Changed

  • PromReporter no longer directly reads txm related status from the db, and instead uses the txStore API.
  • L2Suggested mode is now called SuggestedPrice
  • Console logs will now escape (non-whitespace) control characters
  • Following EVM Pool metrics were renamed:
    • evm_pool_rpc_node_states ā†’ multi_node_states
    • evm_pool_rpc_node_num_transitions_to_alive ā†’ pool_rpc_node_num_transitions_to_alive
    • evm_pool_rpc_node_num_transitions_to_in_sync ā†’ pool_rpc_node_num_transitions_to_in_sync
    • evm_pool_rpc_node_num_transitions_to_out_of_sync ā†’ pool_rpc_node_num_transitions_to_out_of_sync
    • evm_pool_rpc_node_num_transitions_to_unreachable ā†’ pool_rpc_node_num_transitions_to_unreachable
    • evm_pool_rpc_node_num_transitions_to_invalid_chain_id ā†’ pool_rpc_node_num_transitions_to_invalid_chain_id
    • evm_pool_rpc_node_num_transitions_to_unusable ā†’ pool_rpc_node_num_transitions_to_unusable
    • evm_pool_rpc_node_highest_seen_block ā†’ pool_rpc_node_highest_seen_block
    • evm_pool_rpc_node_num_seen_blocks ā†’ pool_rpc_node_num_seen_blocks
    • evm_pool_rpc_node_polls_total ā†’ pool_rpc_node_polls_total
    • evm_pool_rpc_node_polls_failed ā†’ pool_rpc_node_polls_failed
    • evm_pool_rpc_node_polls_success ā†’ pool_rpc_node_polls_success

Removed

  • Removed Optimism2 as a supported gas estimator mode

Fixed

  • Corrected Ethereum Sepolia LinkContractAddress to 0x779877A7B0D9E8603169DdbD7836e478b4624789
  • Fixed a bug that caused the Telemetry Manager to report incorrect health

Upcoming required configuration changes

Starting in v2.9.0:

  • TelemetryIngress.URL and TelemetryIngress.ServerPubKey will no longer be allowed. Any TOML configuration that sets this fields will prevent the node from booting. These fields will be replaced by [[TelemetryIngress.Endpoints]]
  • P2P.V1 will no longer be supported and must not be set in TOML configuration in order to boot. Use P2P.V2 instead. If you are using both, V1 can simply be removed.

Changes in v2.7.2 nodes - 2023-12-14

v2.7.2 release notes

Fixed

  • Fixed a bug that caused nodes without OCR or OCR2 enabled to fail config validation if P2P.V2 was not explicitly disabled. With this fix, NOPs will not have to make changes to their config.

Changes in v2.7.1 nodes

v2.7.1 release notes

Fixed

  • Fixed a bug that causes the node to shutdown if all configured RPC's are unreachable during startup.

Changes in v2.7.0 nodes

v2.7.0 release notes

Added

  • Added new configuration field named LeaseDuration for EVM.NodePool that will periodically check if internal subscriptions are connected to the "best" (as defined by the SelectionMode) node and switch to it if necessary. Setting this value to 0s will disable this feature.

  • Added multichain telemetry support. Each network/chainID pair must be configured using the new fields:

    [[TelemetryIngress.Endpoints]]
Network = '...' # e.g. EVM. Solana, Starknet, Cosmos
ChainID = '...' # e.g. 1, 5, devnet, mainnet-beta
URL = '...'
ServerPubKey = '...'

    These will eventually replace TelemetryIngress.URL and TelemetryIngress.ServerPubKey. Setting TelemetryIngress.URL and TelemetryIngress.ServerPubKey alongside [[TelemetryIngress.Endpoints]] will prevent the node from booting. Only one way of configuring telemetry endpoints is supported.

  • Added bridge_name label to pipeline_tasks_total_finished prometheus metric. This should make it easier to see directly what bridge was failing out from the CL NODE perspective.

  • LogPoller will now use finality tags to dynamically determine finality on evm chains if UseFinalityTags=true, rather than the fixed FinalityDepth specified in toml config

Changed

  • P2P.V1 is now disabled (Enabled = false) by default. It must be explicitly enabled with true to be used. However, it is deprecated and will be removed in the future.
  • P2P.V2 is now enabled (Enabled = true) by default.

Upcoming required configuration changes

Starting in v2.9.0:

  • TelemetryIngress.URL and TelemetryIngress.ServerPubKey will no longer be allowed. Any TOML configuration that sets this fields will prevent the node from booting. These fields will be replaced by [[TelemetryIngress.Endpoints]]
  • P2P.V1 will no longer be supported and must not be set in TOML configuration in order to boot. Use P2P.V2 instead. If you are using both, V1 can simply be removed.

Removed

  • Removed the ability to set a next nonce value for an address through CLI

Changes in v2.6.0 nodes

v2.6.0 release notes

Added

  • Simple password use in production builds is no longer allowed. Nodes with this configuration will not boot and will not pass config validation.
  • Helper migrations function for injecting env vars into goose migrations. This was done to inject chainID into evm chain id not null in specs migrations.
  • OCR2 jobs now support querying the state contract for configurations if it has been deployed. This can help on chains such as BSC which manage state bloat by arbitrarily deleting logs older than a certain date. In this case, if logs are missing we will query the contract directly and retrieve the latest config from chain state. Chainlink Nodes will perform no extra RPC calls unless the job spec has this feature explicitly enabled. On chains that require this, nops may see an increase in RPC calls. This can be enabled for OCR2 jobs by specifying ConfigContractAddress in the relay config TOML.

Removed

  • Removed support for sending telemetry to the deprecated Explorer service. All nodes will have to remove Explorer related keys from TOML configuration and env vars.
  • Removed default evmChainID logic where evmChainID was implicitly injected into the jobspecs based on node EVM chainID toml configuration. All newly created jobs that have the evmChainID field must explicitly define evmChainID in the job spec.
  • Removed keyset migration that migrated v1 keys to v2 keys. All keys should've been migrated by now, and we don't permit creation of new v1 keys anymore

All nodes must remove the following secret configurations:

  • Explorer.AccessKey
  • Explorer.Secret

All nodes must remove the following configuration field: ExplorerURL

Fixed

  • Unauthenticated users executing CLI commands previously generated a confusing error log, which is now removed: [ERROR] Error in transaction, rolling back: session missing or expired, please login again pg/transaction.go:118
  • Fixed a bug that was preventing job runs to be displayed when the job chainID was disabled.
  • chainlink txs evm create returns a transaction hash for the attempted transaction in the CLI. Previously only the sender, recipient and unstarted state were returned.
  • Fixed a bug where evmChainId is requested instead of id or evm-chain-id in CLI error verbatim
  • Fixed a bug that would cause the node to shut down while performing backup
  • Fixed health checker to include more services in the prometheus health metric and HTTP /health endpoint

Changes in v2.5.0 nodes

v2.5.0 release notes

Upcoming Required Configuration Change

  • Starting in 2.6.0, chainlink nodes will no longer allow insecure configuration for production builds. Any TOML configuration that sets the following line will fail validation checks in node start or node validate:
AllowSimplePasswords=true
  • To migrate on production builds, update the database password set in Database.URL to be 16 - 50 characters without leading or trailing whitespace. URI parsing rules apply to the chosen password - refer to RFC 3986 for special character escape rules.

Added

  • Various Functions improvements

Changes in v2.4.0 nodes

v2.4.0 release notes

Fixed

  • Updated v2/keys/evm and v2/keys/eth routes to return 400 and 404 status codes where appropriate. Previously 500s were returned when requested resources were not found or client requests could not be parsed.
  • Fixed withdrawing ETH from CL node for EIP1559 enabled chains. Previously would error out unless validation was overridden with allowHigherAmounts.

Added

  • Added the ability to specify and merge fields from multiple secrets files. Overrides of fields and keys are not allowed.

Upcoming Required Configuration Change

  • Starting in 2.6.0, Chainlink nodes will no longer allow insecure configuration for production builds. Any TOML configuration that sets the following line will fail validation checks in node start or node validate:
AllowSimplePasswords=true
  • To migrate on production builds, update the database password set in Database.URL to be 16 - 50 characters without leading or trailing whitespace. URI parsing rules apply to the chosen password - refer to RFC 3986 for special character escape rules.

Changes in v2.3.0 nodes

v2.3.0 release notes

Added

  • Add a new field called Order (range from 1 to 100) to EVM.Nodes that is used for the PriorityLevel node selector and also as a tie-breaker for HighestHead and TotalDifficulty. Order levels are considered in ascending order. If not defined it will default to Order = 100 (last level).

  • Added new node selection mode called PriorityLevel for EVM, it is a tiered round-robin in ascending order of theOrder field. Example:

    [EVM.NodePool]
SelectionMode = 'PriorityLevel'

[[EVM.Nodes]]
Name = '...'
WSURL = '...'
HTTPURL = '...'
Order = 5

  • The config keys WebServer.StartTimeout and WebServer.HTTPMaxSize. These keys respectively set a timeout for the node server to start and set the max request size for HTTP requests. Previously these attributes were set by JobPipeline.DefaultHTTPLimit/JobPipeline.DefaultHTTPTimeout. To migrate to these new fields, set their values to be identical to JobPipeline.DefaultHTTPLimit/JobPipeline.DefaultHTTPTimeout.

  • Low latency oracle jobs now support in-protocol block range guarantees. This is necessary in order to produce reports with block number ranges that do not overlap. It can now be guaranteed at the protocol level, so we can use local state instead of relying on an unreliable round-trip to the Mercury server.

  • New settings Evm.GasEstimator.LimitJobType.OCR2, OCR2.DefaultTransactionQueueDepth, OCR2.SimulateTransactions for OCR2 jobs. These replace the settings Evm.GasEstimator.LimitJobType.OCR, OCR.DefaultTransactionQueueDepth, and OCR.SimulateTransactions for OCR2.

  • Add new config parameter to OCR and OCR2 named TraceLogging that enables trace logging of OCR and OCR2 jobs, previously this behavior was controlled from the P2P.TraceLogging parameter. To maintain the same behavior set OCR.TraceLogging and OCR2.TraceLogging to the same value P2P.TraceLogging was set.

  • Add two new config parameters WebServer.ListenIP and WebServer.TLS.ListenIP which allows binding Chainlink HTTP/HTTPS servers to a particular IP. The default is '0.0.0.0' which listens to all IP addresses (same behavior as before). Set to '127.0.0.1' to only allow connections from the local machine (this can be handy for local development).

  • Add several new metrics for mercury feeds, related to WSRPC connections:

    • mercury_transmit_timeout_count
    • mercury_dial_count
    • mercury_dial_success_count
    • mercury_dial_error_count
    • mercury_connection_reset_count

Node operators may wish to add alerting based around these metrics.

Fixed

  • Fixed a bug in the nodes xxx list command that caused results to not be displayed correctly

Changed

  • Assumption violations for MaxFeePerGas >= BaseFeePerGas and MaxFeePerGas >= MaxPriorityFeePerGas in EIP-1559 effective gas price calculation will now use a gas price if specified
  • Config validation now enforces protection against duplicate chain ids and node fields per provided TOML file. Duplicates across multiple configuration files are still valid. If you have specified duplicate chain ids or nodes in a given configuration file, this change will error out of all node subcommands.
  • Restricted scope of the Evm.GasEstimator.LimitJobType.OCR, OCR.DefaultTransactionQueueDepth, and OCR.SimulateTransactions settings so they apply only to OCR. Previously these settings would apply to OCR2 as well as OCR. You must use the OCR2 equivalents added above if you want your settings to apply to OCR2.

Removed

  • Legacy chain types Optimism and Optimism2. OptimismBedrock is now used to handle Optimism's special cases.
  • Optimism Kovan configurations along with legacy error messages.

Changes in v2.2.0 nodes

v2.2.0 release notes

Added

  • Added experimental support of runtime process isolation for Solana data feeds. This requires plugin binaries to be installed and configured using the CL_SOLANA_CMD and CL_MEDIAN_CMD environment variables. See the plugins/README.md for more information.

Fixed

  • Fixed a bug which made it impossible to re-send the same transaction after abandoning it while manually changing the nonce.

Changed

  • Set the default for EVM.GasEstimator.BumpTxDepth to EVM.Transactions.MaxInFlight.
  • Bumped batch size defaults for EVM-specific configurations. If you are overriding any of these fields in your local config, consider if it is necessary:
    • LogBackfillBatchSize = 1000
    • RPCDefaultBatchSize = 250
    • GasEstimator.BatchSize = 25
  • Dropped support for the Development Mode configuration. The CL_DEV environment variable is now ignored on production builds.
  • Updated the Docker image's PostgreSQL client (used for backups) to v15 in order to support PostgreSQL v15 servers.

Changes in v2.1.0 nodes

v2.1.0 release notes

  • The chainlink db CLI commands now validate the TOML configuration and secrets before executing. These commands will report errors if any database-specific configuration is invalid.

Changes in v2.0.0 nodes

v2.0.0 release notes

Added

  • Add OCR2 Plugin selection for FMS
  • Added kebab case aliases for the following flags:
    • evm-chain-id alias for evmChainID in commands: chainlink blocks replay, chainlink forwarders track, chainlink keys ... chain
    • old-password alias for oldpassword in commands: chainlink keys ... import
    • new-password alias for newpassword in commands: chainlink keys ... export
    • new-role alias for newrole in commands: admin users chrole
    • set-next-nonce alias for setNextNonce in commands: chainlink keys ... chain

Changed

  • TOML configuration and secrets are now scoped to chainlink node command rather than being global flags.
  • TOML configuration validation has been moved from chainlink config validate to chainlink node validate.
  • Move chainlink node {status,profile} to chainlink admin {status,profile}.

Removed

  • Configuration with legacy environment variables is no longer supported. TOML is required.

Changes in v1.13.0 nodes

v1.13.0 release notes

TOML Configuration

TOML configuration for Chainlink nodes is stable and recommended for mainnet deployments. TOML configuration will be the only supported configuration method starting with v2.0.0. Enable TOML configuration by specifying the -config FILENAME.toml flag with the path to your TOML file. Alternatively, you can specify the raw TOML config in the CL_CONFIG environment variable. See the CONFIG.md and SECRETS.md on GitHub to learn more.

Added

  • Added support for sending OCR2 job specs to the Feeds Manager.
  • Log poller filters are now saved in the database and restored on node startup to guard against missing logs during periods where services are temporarily unable to start.

Updated

  • TOML config: The environment variable CL_CONFIG is always processed as the last configuration. This has the effect of being the final override for any values provided via configuration files.

Changed

  • The Feeds Manager is now enabled by default.

Removed

  • Terra is no longer supported.

Changes in v1.12.0 nodes

v1.12.0 release notes

Added

  • Prometheus gauge mailbox_load_percent for percent of "Mailbox" capacity used.
  • New config option JobPipeline.MaxSuccessfulRuns caps the total number of saved completed runs per job. This is done in response to the pipeline_runs table potentially becoming large, which can cause performance degradation. The default is set to 10,000. You can set it to 0 to disable run saving entirely. NOTE: This can only be configured via TOML and not with an environment variable.
  • Prometheus gauge vector feeds_job_proposal_count to track counts of job proposals partitioned by proposal status.
  • Support for variable expression for the minConfirmations parameter on the ethtx task.

Updated

  • Removed the KEEPER_TURN_FLAG_ENABLED as all networks and nodes have switched this to true. The variable should be completely removed.
  • Removed the Keeper.UpkeepCheckGasPriceEnabled config and the KEEPER_CHECK_UPKEEP_GAS_PRICE_FEATURE_ENABLED environment variable. This feature is deprecated and the variable should be completely removed.

Fixed

  • Fixed (SQLSTATE 42P18) error on Job Runs page, when attempting to view specific older or infrequenty run jobs.
  • The config dump subcommand was fixed to dump the correct config data. The P2P.V1.Enabled config logic incorrectly matched V2, by only setting explicit true values so that otherwise the default is used. The V1.Enabled default value is actually true already, and is now updated to only set explicit false values.

Changes in v1.11.0 nodes

v1.11.0 release notes

Added

  • Added a new mode for the NODE_SELECTION_MODE environment variable. Use TotalDifficulty to select the node with the greatest total difficulty.
  • Added the NODE_SYNC_THRESHOLD environment variable to ensure that live nodes do not lag too far behind.
  • Added the BRIDGE_CACHE_TTL environment variable which caches bridge responses for a specified amount of time.
  • Add the prometheus metrics labelled by bridge name for monitoring external adapter queries. The following metrics are included:
    • bridge_latency_seconds
    • bridge_errors_total
    • bridge_cache_hits_total
    • bridge_cache_errors_total
  • āš ļø Experimental: āš ļø Added static configuration using TOML files as an alternative to the existing combination of environment variables and persisted database configurations. For this release, use TOML for configuration only on test networks. In the future with v2.0.0, TOML configuration will become the only supported configuration method. Enable TOML configuration by specifying the -config FILENAME.toml flag with the path to your TOML file. Alternatively, you can specify the raw TOML config in the CL_CONFIG environment variable. See the CONFIG.md and SECRETS.md on GitHub to learn more.

Fixed

  • Fixed a minor bug where Chainlink would not always resend all pending transactions when using multiple keys.

Updated

  • NODE_NO_NEW_HEADS_THRESHOLD=0 no longer requires NODE_SELECTION_MODE=RoundRobin.

Changes in v1.10.0 nodes

v1.10.0 release notes

Added

Changed

  • The default maximum gas price on most networks is now effectively unlimited.
  • If the EVMChainID is not set explicitly in the job spec for a new OCR job, the field is now automatically added with a default chain ID.
    • Old OCR jobs missing EVMChainID continue to run on any chain that the ETH_CHAIN_ID variable is set to (or the first chain if it is not set). This can be changed after a restart.
    • Newly created OCR jobs run only on a single fixed chain and are unaffected by changes to ETH_CHAIN_ID after the job is added.
    • It should no longer be possible to end up with multiple OCR jobs for a single contract running on the same chain. Only one job per contract per chain is allowed.
    • If there are any existing duplicate jobs per contract and per chain, all but the jobs with the latest creation date will be pruned during the upgrade.

Fixed

  • Fixed minor bug where Chainlink would attempt and fail to estimate a tip cap higher than the maximum configured gas price in EIP1559 mode. It now caps the tipcap to the max instead of erroring.
  • Fixed a bug where it was impossible to remove ETH keys that had extant transactions. Now, removing an ETH key will drop all associated data automatically, including past transactions.

Automatic connectivity detection

Chainlink will no longer bump excessively if the network is broken.

This feature only applies on EVM chains when using BlockHistoryEstimator (the most common case). Chainlink will now try to automatically detect if there is a transaction propagation/connectivity issue and prevent bumping in these cases. This can help avoid the situation where RPC nodes are not propagating transactions for some reason (e.g., go-ethereum bug, networking issue ...etc) and Chainlink responds in a suboptimal way by bumping transactions to a very high price in an effort to get them mined. This can lead to unnecessary expense when the connectivity issue is resolved and the transactions are finally propagated into the mempool.

This feature is enabled by default with fairly conservative settings: if a transaction has been priced above the 90th percentile of the past 12 blocks, but still wants to bump due to not being mined, a connectivity/propagation issue is assumed and all further bumping will be prevented for this transaction. In this situation, Chainlink will start firing the block_history_estimator_connectivity_failure_count prometheus counter and logging at critical level until the transaction is mined.

The default settings should work fine for most users. For advanced users, the values can be tweaked by changing BLOCK_HISTORY_ESTIMATOR_CHECK_INCLUSION_BLOCKS and BLOCK_HISTORY_ESTIMATOR_CHECK_INCLUSION_PERCENTILE.

To disable connectivity checking completely, set BLOCK_HISTORY_ESTIMATOR_CHECK_INCLUSION_BLOCKS=0.

Changes in v1.9.0 nodes

v1.9.0 release notes

  • Added the length task and the lessthan task for jobs.
  • Added the gasUnlimited parameter to the ethcall task.
  • The Keys page in Operator UI includes several admin commands that were previously available only by using the keys eth chain commands:
    • Ability to abandon all current transactions: This is the same as the abandon CLI command. Previously it was necessary to edit the database directly to abandon transactions. This command makes it easier to resolve issues that require transactions to be abandoned.
    • Ability to enable/disable a key for a specific chain: This allows you to control keys on a per-chain basis.
    • Ability to manually set the nonce for a key. This gives you a way to set the next nonce for a specific key in the UI, which can be useful for debugging.

Changes in v1.8.1 nodes

v1.8.1 release notes

  • Added several improvements for Arbitrum Nitro including a multi-dimensional gas model, with dynamic gas pricing and limits.
    • The new default estimator for Arbitrum networks uses the suggested gas price up to ETH_MAX_GAS_PRICE_WEI with a 1000 gwei default value and an estimated gas limit up to ETH_GAS_LIMIT_MAX with a 1,000,000,000 default.
    • Remove the GAS_ESTIMATOR_MODE environment variable to use the new defaults.
  • ETH_GAS_LIMIT_MAX to puts a maximum on the gas limit returned by the Arbitrum estimator.

Changes in v1.8.0 nodes

v1.8.0 release notes

Added

  • Added the hexencode and base64encode tasks (pipeline). See the Hex Encode Task and Base64 Encode Task pages for examples.
  • forwardingAllowed per job attribute to allow forwarding txs submitted by the job.
  • Added Arbitrum Goerli configuration support.
  • Added the NODE_SELECTION_MODE (EVM.NodePool.SelectionMode) environment variable, which controls node picking strategy. Supported values are:
    • HighestHead is the default mode, which picks a node that has the highest reported head number among other alive nodes. When several nodes have the same latest head number, the strategy sticks to the last used node. This mode requires NODE_NO_NEW_HEADS_THRESHOLD to be configured, otherwise it will always use the first alive node.
    • RoundRobin mode iterates among available alive nodes. This was the default behavior prior to this release.
  • New evm keys chain command. This can also be accessed at /v2/keys/evm/chain. This command has the following uses:
    • Manually set or reset a nonce: chainlink evm keys chain --address "0xEXAMPLE" --evmChainID 99 --setNextNonce 42
    • Enable a key for a particular chain: chainlink evm keys chain --address "0xEXAMPLE" --evmChainID 99 --setEnabled true
    • Disable a key for a particular chain: chainlink evm keys chain --address "0xEXAMPLE" --evmChainID 99 --setEnabled false
  • It is now possible to use the same key across multiple chains.

Changed

  • The chainlink admin users update command is replaced with chainlink admin users chrole. Only the role can be changed for a user.
  • Keypath now supports paths with any depth, instead of limiting it to 2.
  • Arbitrum chains are no longer restricted to only FixedPrice GAS_ESTIMATOR_MODE.
  • Updated Arbitrum Rinkeby & Mainnet configurations for Nitro.

Removed

  • The setnextnonce local client command is removed and is replaced by the more general evm keys chain command client command.

Changes in v1.7.1 nodes

v1.7.1 release notes

Fixed

  • Arbitrum Nitro client error support

Changes in v1.7.0 nodes

v1.7.0 release notes

Added

  • p2pv2Bootstrappers is added as a new optional property of OCR1 job specs. The default can still be specified with the P2PV2_BOOTSTRAPPERS environment variable.

  • Added official support for the Sepolia testnet on Chainlink nodes.

  • Added hexdecode task and the base64decode task (pipeline).

  • Added support for the Besu execution client. Although Chainlink supports Besu, Besu itself has several issues that can make it unreliable. For additional context, see the following issues:

  • Added Multi-user and Role Based Access Control functionality. This allows the root admin CLI user and additional admin users to create and assign tiers of role-based access to new users. These new API users are able to log in to the Operator UI independently and can each have specific roles tied to their account. There are four roles: admin, edit, run, and view.

    • User management can be configured through the use of the new admin CLI command chainlink admin users. Be sure to run chainlink admin login. For example, a readonly user can be created with: chainlink admin users create --email=operator-ui-read-only@test.com --role=view.
    • Updated documentation repo with a break down of actions to required role level

  • Added gas limit control for individual job specs and individual job types. The following rule of precedence is applied:

    1. The task-specific parameter gasLimit overrides anything else when specified. For example, the ethtx task has a gasLimit parameter that overrides the other defaults for this specific task.

    2. The job-spec attribute gasLimit applies only to a specific job spec.

    3. The job-type limits affect any jobs of the corresponding type. The following environment variables are available:

    4. The global ETH_GAS_LIMIT_DEFAULT (EVM.GasEstimator.LimitDefault) value is used only when the preceding rules are not set.

Fixed

  • Addressed a very rare bug where using multiple nodes with differently configured RPC tx fee caps could cause missed transactions. Ensure that your RPC nodes have no caps. For more information, see the performance and tuning guide.
  • Improved handling of unknown transaction error types to make Chainlink more robust in certain cases on unsupported chains or RPC clients.

Changes in v1.6.0 nodes

v1.6.0 release notes

  • Simplified password complexity requirements. All passwords used with Chainlink must meet the following requirements:
    • Must be 16 characters or more
    • Must not contain leading or trailing whitespace
    • User passwords must not contain the user's API email
  • Simplified the Keepers job spec by removing the observation source from the required parameters.

Changes in v1.5.0 nodes

v1.5.0 release notes

  • Chainlink will not boot if the Postgres database password is missing or insecure. Passwords must conform to the following rules:

    • Must be longer than 12 characters
    • Must comprise at least 3 of the following items:
      • Lowercase characters
      • Uppercase characters
      • Numbers
      • Symbols
    • Must not comprise:
      • More than three identical consecutive characters
      • Leading or trailing whitespace (note that a trailing newline in the password file, if present, will be ignored)

    For backward compatibility, you can bypass this check at your own risk by setting SKIP_DATABASE_PASSWORD_COMPLEXITY_CHECK=true.

  • The following environment variables are deprecated and will be removed in a future release. They are replaced by the following command line arguments:

    • INSECURE_SKIP_VERIFY: Replaced by the --insecure-skip-verify CLI argument
    • CLIENT_NODE_URL: Replaced by the --remote-node-url URL CLI argument
    • ADMIN_CREDENTIALS_FILE: Replaced by the --admin-credentials-file FILE CLI argument

    Run ./chainlink --help to learn more about these CLI arguments.

  • The Optimism2 GAS_ESTIMATOR_MODE has been renamed to L2Suggested. The old name is still supported for now.

  • The p2pBootstrapPeers property on OCR2 job specs has been renamed to p2pv2Bootstrappers.

Added

  • Added the ETH_USE_FORWARDERS config option to enable transactions forwarding contracts.

  • In the directrequest job pipeline, three new block variables are available:

    • $(jobRun.blockReceiptsRoot) : the root of the receipts trie of the block (hash)
    • $(jobRun.blockTransactionsRoot) : the root of the transaction trie of the block (hash)
    • $(jobRun.blockStateRoot) : the root of the final state trie of the block (hash)

  • ethtx tasks can now be configured to error if the transaction reverts onchain. You must set failOnRevert=true on the task to enable this behavior:

    foo [type=ethtx failOnRevert=true ...]

    The ethtx task now works as follows:

    • If minConfirmations == 0, task always succeeds and nil is passed as output.
    • If minConfirmations > 0, the receipt is passed through as output.
    • If minConfirmations > 0 and failOnRevert=true then the ethtx task will error on revert.
    • If minConfirmations is not set on the task, the chain default will be used which is usually 12 and always greater than 0.

  • http task now allows specification of request headers. Use it like the following example:

    foo [type=http headers="[\\"X-Header-1\\", \\"value1\\", \\"X-Header-2\\", \\"value2\\"]"].

Fixed

  • Fixed max_unconfirmed_age metric. Previously this would incorrectly report the max time since the last rebroadcast, capping the upper limit to the EthResender interval. This now reports the correct value of total time elapsed since the first broadcast.

  • Correctly handle the case where bumped gas would exceed the RPC node's configured maximum on Fantom. Note that node operators should check their Fantom RPC node configuration and remove the fee cap if there is one.

  • Fixed handling of the Metis internal fee change.

Removed

  • The Optimism OVM 1.0 GAS_ESTIMATOR_MODE has been removed and the Optimism2 GAS_ESTIMATOR_MODE has been renamed to L2Suggested.

  • MIN_OUTGOING_CONFIRMATIONS has been removed and no longer has any effect. The ETH_FINALITY_DEPTH environment variable is now used as the default for ethtx confirmations instead. You can override this on a per-task basis by setting minConfirmations in the task definition. For example, foo [type=ethtx minConfirmations=42 ...].

    This setting might have a minor impact on performance for very high throughput chains. If you don't care about reporting task status in the UI, set minConfirmations=0 in your job specs. For more details, see the Optimizing EVM Performance page.

Changes in v1.4.1 nodes

v1.4.1 release notes

  • Added a fix to ensure that a failed EthSubscribe does not register (*rpc.ClientSubscription)(nil), which leads to a panic when unsubscribing.
  • Fix parsing of float values on job specs.

Changes in v1.4.0 nodes

v1.4.0 release notes

  • JSON parse tasks in TOML now support a custom separator parameter to substitute for the default ,.
  • Slow SQL queries are now logged.
  • Updated the block explorer URLs to include FTMScan and SnowTrace.
  • Keeper upkeep order can now be shuffled.
  • Several fixes. See the release notes for a full list of changes.

Changes in v1.3.0 nodes

v1.3.0 release notes

  • Added disk rotating logs. See the Node Logging and LOG_FILE_MAX_SIZE documentation for details.
  • Added support for the force flag on the chainlink blocks replay CLI command. If set to true, already consumed logs that would otherwise be skipped will be rebroadcasted.
  • Added a version compatibility check when using the CLI to login to a remote node. The bypass-version-check flag skips this check.
  • Changed default locking mode to "dual". See the DATABASE_LOCKING_MODE documentation for details.
  • Specifying multiple EVM RPC nodes with the same URL is no longer supported. If you see ERROR 0106_evm_node_uniqueness.sql: failed to run SQL migration, you have multiple nodes specified with the same URL and you must fix this before proceeding with the upgrade.
  • EIP-1559 is now enabled by default on the Ethereum Mainnet. See the EVM_EIP1559_DYNAMIC_FEES documentation for details.
  • Added new Chainlink Automation feature that includes gas price in calls to checkUpkeep(). To enable the feature, set KEEPER_CHECK_UPKEEP_GAS_PRICE_FEATURE_ENABLED to true. Use this setting only on Polygon networks.

Changes in v1.2.0 nodes

v1.2.0 release notes

Significant changes:

  • Added support for the Nethermind Ethereum client.
  • Added support for batch sending telemetry to the ingress server to improve performance.
  • New environment variables: See the release notes for details.
  • Removed the deleteuser CLI command.
  • Removed the LOG_TO_DISK environment variable.

See the v1.2.0 release notes for a complete list of changes and fixes.

Changes in v1.1.0 nodes

v1.1.0 release notes

The v1.1.0 release includes several substantial changes to the way you configure and operate Chainlink nodes:

  • Legacy environment variables: Legacy environment variables are supported, but they might be removed in future node versions. See the Configuring Chainlink Nodes page to learn how to migrate your nodes away from legacy environment variables and use the API, CLI, or GUI exclusively to administer chains and nodes.
  • Full EIP1559 Support: Chainlink nodes include experimental support for submitting transactions using type 0x2 (EIP-1559) envelope. EIP-1559 mode is off by default, but can be enabled either globally or on a per-chain basis.
  • New log level added:
    • [crit]: Critical level logs are more severe than [error] and require quick action from the node operator.
  • Multichain support (Beta): Chainlink now supports connecting to multiple different EVM chains simultaneously. This is disabled by default. See the v1.1.0 Changelog for details.

With multliple chain support, eth node configuration is stored in the database.

The following environment variables are DEPRECATED:

  • ETH_URL
  • ETH_HTTP_URL
  • ETH_SECONDARY_URLS

Setting ETH_URL will cause Chainlink to automatically overwrite the database records with the given ENV values every time Chainlink boots. This behavior is used mainly to ease the process of upgrading from older versions, and on subsequent runs (once your old settings have been written to the database) it is recommended to unset these ENV vars and use the API commands exclusively to administer chains and nodes.

If you wish to continue using these environment variables (as it used to work in 1.0.0 and below) you must ensure that the following are set:

  • ETH_CHAIN_ID (mandatory)
  • ETH_URL (mandatory)
  • ETH_HTTP_URL (optional)
  • ETH_SECONDARY_URLS (optional)

If, instead, you wish to use the API/CLI/GUI to configure your chains and eth nodes (recommended) you must REMOVE the following environment variables:

  • ETH_URL
  • ETH_HTTP_URL
  • ETH_SECONDARY_URLS

This will cause Chainlink to use the database for its node configuration.

NOTE: ETH_CHAIN_ID does not need to be removed, since it now performs the additional duty of specifying the default chain in a multichain environment (if you leave ETH_CHAIN_ID unset, the default chain is simply the "first").

For more information on configuring your node, check the configuration variables in the docs.

Before you upgrade your nodes to v1.1.0, be aware of the following requirements:

  • If you are upgrading from a previous version, you MUST first upgrade the node to at least v0.10.15.
  • Always take a Database snapshot before you upgrade your Chainlink nodes. You must be able to roll the node back to a previous version in the event of an upgrade failure.

See the v1.1.0 release notes for a complete list of changes and fixes.

Changes in v1.0.0 and v1.0.1 nodes

v1.0.0 release notes v1.0.1 release notes

Before you upgrade your nodes to v1.0.0 or v1.0.1, be aware of the following requirements:

  • If you are upgrading from a previous version, you MUST first upgrade the node to at least v0.10.15.
  • Always take a Database snapshot before you upgrade your Chainlink nodes. You must be able to roll the node back to a previous version in the event of an upgrade failure.

What's next

Get the latest Chainlink content straight to your inbox.