In one of the projects I work, there was a requirement to schedule only one job for a record at a time. If someone wants to schedule another one for the same record, the system should clear the old schedule for it and schedule the new one.
To do this, we are going to add some helpers to app/workers/base_worker.rb
using Sidekiq API.
Be careful with the following method when dealing with 10,000+ jobs, you could look at Sidekiq Pro for efficiency.
Add the following helper
This helper deletes the jobs from sheduled set that match with the worker name and the arguments. Tweak the following helper as per the requirements to optimize the performance.
def self.delete_matched(worker_name, arguments)
set = Sidekiq::ScheduledSet.new
set = if arguments.is_a?(Array) && arguments.first.is_a?(String)
set.scan arguments.first
elsif arguments.is_a?(String)
set.scan arguments
else
set.scan worker_name
end
set.each do |job|
job.delete if job.display_class == worker_name && job.args == [arguments]
end
end
Sidekiq provides perform_async,
we can add following helpers around this method.
Use perform_unique_async
This method enqueues a unique job
##
# Usage:
# HeartbeatsWorker.perform_unique_async(id)
#
def self.perform_unique_async(*arguments)
self.delete_matched(name, arguments)
self.perform_async(arguments)
end
Use perform_unique_in
This method schedules a unique job to be performed in given time.
##
# Usage:
# HeartbeatsWorker.perform_unique_in(30.seconds, id)
#
def self.perform_unique_in(t, *arguments)
self.delete_matched(name, arguments)
self.perform_in(t, arguments)
end
Use perform_unique_at
This method schedules a unique job to be performed at a given time.
##
# Usage:
# HeartbeatsWorker.perform_unique_at(datetime, id)
#
def self.perform_unique_at(t, *arguments)
self.delete_matched(name, arguments)
self.perform_at(t, arguments)
end
Here is the full BaseWorker class
Finally,
In this post, we explored how we could remove duplicate jobs from the Sidekiq’s ScheduledSet and enqueue unique ones. You can dig deeper into the Sidekiq API to customize it as per the needs.