How to limit record management functions in SP2010 to a specific group of Records Managers?

In the SharePoint 2010 settings for (in-place) Records Management you can configure who can Declare / Undeclare records.
However you can only select from these three options:

• All list contributors and administrators
  Any user with the “edit items” permissions to a list can declare and undeclared items as records.
• Only list administrators
  Only users with “manage list” permissions to a list can declare and undeclare items as records.
• Only policy actions
  Only policy actions or custom code running as the System Account can declare and undeclare items as records.

So what happens if there is a centralized group of records managers inside a company? In other words: how can you limit the permission to Declare/Undeclare records to a specific group of SharePoint users?