Today I Learned

I’ve been building a system that imports a lot of records from another system. The items from the other system have stable identifiers, but many other attributes will change.

In the past I’ve handled this scenario by iterating on each record, and individually “upsetting” into the local database. It would look something like

entries = RemoteSystem.new.entries
entries.each do |entry|
  attributes = entry_attributes_from_remote_system(entry)
  Entry.find_or_initialize_by(identifier: attributes.fetch("identifier").update(attributes)
end

This works fine and good, but I learned there’s a better way! Rails 6 introduced upsert_all. With this new method, many round trips to the database are replaced with one command.

entries = RemoteSystem.new.entries
entries_attributes = entries.map { |entry| entry_attributes_from_remote_system(entry) }
Entry.upsert_all(entries_attributes, unique_by: %w[identifier])

The one gotcha I’ve found so far, is that it fails if entries_attributes is an empty array. Easily mitigated with a presence check.

Entry.upsert_all(entries_attributes, unique_by: %w[identifier]) unless entries_attributes.blank?

The Big Binary blog post has a lot of great information on usage.