Why isn't my connection using the Bulk API to write to Salesforce?

Issue

My connection is using SOAP to write to Salesforce instead of Bulk, which is much slower. How can I speed this up?

Resolution

See SOAP vs Bulk API for details on the conditions for Bulk and SOAP API usage.

Heroku Connect will try to use Salesforce's Bulk API, if all the conditions are met, when there are more than 2,000 contiguous changes of the same type for a given object. For example, more than 2,000 contiguous inserts for a specific object.

To confirm whether your database write patterns generate enough consecutive changes for the same type (INSERT, UPDATE or DELETE) to the same mapping, use the following query:

Notes:

• This query can take a long time to run when your connection has many pending changes in the _trigger_log table.
• If your connection uses a custom schema instead of the default salesforce one, replace salesforce._trigger_log to <schema>._trigger_log in the query.

SELECT
  min(table_name) AS mapping,
  min(action) AS action, 
  count(*) AS consecutive_rows,
  min(created_at) AS first_created_at
FROM (
  SELECT
    t1.*,
    sum(group_flag) OVER (order by created_at) as grp
  FROM (
    SELECT
      *,
      CASE
          WHEN lag(table_name) OVER (order by created_at) = table_name 
               AND lag(action) OVER (ORDER BY created_at) = action
          THEN null
          ELSE 1
      END as group_flag
    FROM salesforce._trigger_log
  ) t1
) t2
GROUP BY grp
ORDER BY first_created_at;