Filtering OpenSearch Plugins with Systemd

Filtering OpenSearch Plugins with Systemd

Have you installed OpenSearch via Deb or RPM? Check your plugins: you have 26 along for the ride, whether you need them or not. This article describes how to selectively enable plugins using systemd. Obviously this doesn't apply to Docker, but there's an easier solution there: make a custom image that removes /usr/share/opensearch/plugins.

Why would I want to remove plugins?

At work, I help manage several very large (50+ nodes) OpenSearch clusters. Performance is critical, and unused plugins take up memory. We also have several of our own custom plugins. We don't know if the pre-installed plugins could interact with our custom plugins, and we don't want to find out.

Can't you just remove the plugins you don't want?

OpenSearch does have a utility called opensearch-plugin, and it can be used to remove unused plugins, as described here. However, this approach doesn't scale because:

  • The utility has prompts, and won't let you remove plugins that are dependencies of other plugins.
  • It's scoped to a single node. There is no cluster-wide way to manage plugins.
  • If you remove the plugin, it'll be deleted from /usr/share/opensearch/plugins. But when you update the package, it will be reinstalled.

There's got to be a better way!

Enter systemd

As described on systemd's website,

TemporaryFileSystem is used to set up ... a new file system namespace [...]for executed processes, and a temporary file system is mounted on each mount point. This is useful to hide files or directories not relevant to the processes invoked by the unit, while necessary files or directories can be still accessed by combining with BindPaths= or BindReadOnlyPaths=:.

You can probably guess where I'm going with this, but here is how we ended up allowlisting the plugins we want with a systemd override (templated with jinja2):

[Service]
# Replace plugins dir with an empty tmpfs
TemporaryFileSystem=/usr/share/opensearch/plugins:rw

BindPaths=
{% for plugin in allowed_plugins %}
BindPaths=/usr/share/opensearch/plugins/{{ plugin }}
{% endfor %}

And then we fill in allowed_plugins with a list:

- opensearch-extra-analysis-turkish
- opensearch-extra-analysis-ukrainian
- opensearch-ltr
- repository-s3

Manually Creating/Applying the Systemd Override

To do this manually, create /etc/systemd/system/opensearch.service.d/override.conf with the above content:

[Service]
# Replace plugins dir with an empty tmpfs
TemporaryFileSystem=/usr/share/opensearch/plugins:rw

BindPaths=/usr/share/opensearch/plugins/opensearch-extra-analysis-turkish
BindPaths=/usr/share/opensearch/plugins/opensearch-extra-analysis-ukrainian
BindPaths=/usr/share/opensearch/plugins/opensearch-ltr
BindPaths=/usr/share/opensearch/plugins/repository-s3

Reload systemd and restart the OpenSearch service (systemd daemon-reload; systemctl restart opensearch)

I Tried That, and the OpenSearch Service Failed to Start

Some plugins are dependencies of other plugins. If you accidentally removed an allowed plugin's dependency, OpenSearch won't load. Check the OpenSearch logs, they are pretty clear when this happens.

Verifying that it worked

Count the number of plugins detected before and after the changes:

curl -s  https://opensearch.service.rl:9200/_cat/plugins | awk '{print $2}' | sort | uniq | wc -l

If there's no difference, make sure your systemd override is activated (systemd daemon-reload) and that you have restarted the OpenSearch service after activating the override.

Congrats! You now have the ability to choose which plugins to load in OpenSearch!

Pages