To achieve this, after SPM client is installed, open /opt/spm/properties/spm-sender.properties file and just add desired hostname to the value of property spm_sender_hostname_alias, e.g. spm_sender_hostname_alias=web1.production.

After that, restart SPM monitor with:

sudo service spm-monitor restart

and restart any Java process which was using javaagent/in-process version of SPM monitor.

Few notes to keep in mind:

• you can do this change at any point, even after SPM client was running on your machine for the past few months or years - you just have to upgrade SPM client to version 1.17.7 or greater
• old data will still be seen in SPM under the old hostname, while new data (after hostname change) will be displayed under the new hostname
• if you are installing SPM client on some machine for the first time, and you want to be 100% sure its original hostname never leaves your network, define your hostname alias in spm-sender.properties file immediately after you complete "1. Package installation" step and before you begin with "2. Client configuration setup" step described on page http(Installation instructions can be accessed from https://apps.sematext.com/spm-reports/client.do.ui/monitoring, click Actions->Install Monitor on app you are installing)

Can I reduce the amount of logs generated by SPM monitor

A: Yes, starting with 1.20.6 version of SPM client you can make SPM monitor reduce the number of messages it logs every minute (otherwise it logs detailed info about its monitoring lifecycle). To do that, change properties files of your SPM applications. You can find those files with:

ls /opt/spm/spm-monitor/conf/spm-monitor-config-*.properties

You can adjust one or more of them, depending on which application's monitor log output you want to reduce. At the bottom of those files add the following line:

SPM_MONITOR_LOGGING_LEVEL=reduced

After you have made the changes, restart your SPM monitor – if you are using standalone variant, run: sudo service spm-monitor restart , otherwise restart your application/java process.

How frequently does SPM monitor collects metrics and can I adjust that interval

SPM monitor by default collects metrics every 10 seconds. To reduce this frequency to 30 seconds, for example, simply add the following line to your SPM monitor properties files located in /opt/spm/spm-monitor/conf :

SPM_MONITOR_COLLECT_INTERVAL=30000

The value is expressed in milliseconds. If you are adjusting it, we recommend setting it to 30000. With bigger values it is possible some 1-minute intervals would be displayed without the data in the UI.

After you make this change, make sure SPM monitor is restarted (if using in-process monitor, restart the monitored services; if using standalone monitor, just run sudo service spm-monitor restart).

How much memory is standalone SPM monitor using and can it be adjusted

By default, each standalone SPM monitor process is started with "-Xmx384m" setting. This means that its JVM heap will use 384 MB at most. In many cases SPM monitor doesn't actually need or use that much memory. If you want to be absolutely sure about it, simply lower this number in /opt/spm/spm-monitor/bin/spm-monitor-starter.sh, with the following variable (around line 63):

JAVA_OPTIONS="$JAVA_OPTIONS -Xmx384m -Xms128m -Xss256k"

You can change "-Xmx" part to "-Xmx192m" for example and see how it goes (check if SPM monitor is still working over the next few hours/days and is still not using much CPU).

We don't recommend changing this setting if you are monitoring Elasticsearch or Solr clusters with a high number of indices or shards. In those cases SPM monitor has to parse and gather large amounts of metrics data returned by Elasticsearch or Solr.  You can contact us in chat or via email (spm-support@sematext.com) if you'd like to get more info and help around making this adjustment.

Can I use SPM to monitor multiple applications running on the same server / VM

A: Yes. There are really 2 different scenarios here:

1) If each of those applications should be monitored under a different SPM application (e.g., you could have Solr running on your server along with some Java app and you want to monitor both - Solr would be monitored with SPM application of Solr or SolrCloud type, while the Java app would be monitored with SPM application of JVM type), just complete all installation steps from http://apps.sematext.com/spm-reports/client.do for each of them separately.

2) If you want them monitored under the same SPM application (e.g., you have 3 Solr instances running on a server), you must use different JVM name for each of them. To do this, "1. Package installation" step on http://apps.sematext.com/spm-reports/client.do should be run only once on this machine, while "2. Client configuration setup" step should be run once for each of the 3 Solr instances. When running script /opt/spm/bin/spm-client-setup-conf.sh in step "2. Client configuration setup", you should add jvmname parameter (and value) at the end of parameter list, like this:

sudo bash /opt/spm/bin/spm-client-setup-conf.sh 11111111-1111-1111-1111-111111111111 solr javaagent jvmname:solr1

In this example, we are setting up things for 3 separate Solr processes, monitored under JVM names: solr1, solr2 and solr3 (you choose any names), so this is adjusted command for solr1 instance.

In the remaining sub-steps of "2. Client configuration setup", replace word "default" with the jvmname value you just used.

"2. Client configuration setup" step will have to be repeated N times, once for each monitored application (in our example 3 times with 3 different jvmname values).

Note: By using this kind of setup, you will be able to see JVM stats of all 3 processes separately (JVM name filter is used to do the filtering). When it comes to other, non-JVM stats, they will be aggregated into single value (for instance, request rate chart will show sum of request rate value of these 3 Solr instances). If you want to see even non-JVM stats separately, you will have to create 3 separate SPM Solr applications (one for each Solr instance running on this machine).

Can I use SPM to monitor my service which runs on Windows or Mac OS X

A: SPM client installers currently exist only for Linux, however there is still a way to monitor your service. If you are OK with installing SPM client on separate linux box, and pointing it (more about that further below) to your service (which should be monitored) running on Windows or Mac, you can use SPM to monitor all non-OS related metrics. That means you will not be able to see CPU or system Memory metrics, but metrics specific to your process will be displayed (for example, in case of Solr, you will see all search, index, cache metrics along with all JVM metrics like pool memory, GC stats, JVM threads etc; as another example, in case of MySQL you will see all metrics related to connections, users, queries, handlers, commands, MyISAM, InnoDB, MySQL traffic, etc) .

When monitoring Solr, Elasticsearch, HBase, Hadoop and other Java-based services, you will have an option to choose between using In-process (javaagent) or Standalone monitor. The workaround described here requires the use of standalone monitor variant. Here's what you'd need to do to see your metrics in SPM:

1. Install spm-client on any Linux box (you can use this box for anything else, it is needed here just to run a process which collects metrics and sends them to Sematext), following steps from http://apps.sematext.com/spm-reports/client.do . In step 1 choose the "Other" tab to create a minimal installation.  This will not install modules needed for monitoring OS metrics - if those modules were installed, they would collect OS metrics of your Linux box, which is probably something you don't need. If you decide to follow instructions from some other tab, keep in mind OS metrics displayed in SPM UI will not be OS metrics of your Windows/Mac machine.
2. In step 2, if you are given a choice between In-process and Standalone monitor, choose Standalone monitor. It will use remote JMX to collect metrics from your Windows/Mac machine. Just follow instructions given on Standalone tab.  The only difference you will want to make is in -Dspm.remote.jmx.url parameter used in monitor properties.  You will know where to adjust it if you follow standard instructions for Standalone monitor. While this parameter will usually have a value like -Dspm.remote.jmx.url=localhost:3000, in this case you will have to replace localhost with the address of your Windows/Mac machine(s) by which it can be reached from your helper Linux box.
3. If you are monitoring something that doesn't offer a choice between In-process and Standalone monitor, installation instructions at http://apps.sematext.com/spm-reports/client.do will explain where you can define your own address. Instructions typically use localhost, but you can instead use something like 10.1.2.3 or my-win-solr-server.mycompany.com to point to the machine that hosts the service you intend to monitor.

In case you want to monitor multiple machines belonging to the same cluster in this way, you can still use SPM installed on a single Linux helper box. Just do this:

• Create a separate SPM application in SPM UI (https://apps.sematext.com/spm-reports/registerApplication.do) for each of the machines which should be monitored
• For each of those SPM applications (and your monitored machines), go through installation process described in http://apps.sematext.com/spm-reports/client.do . Note: Since each SPM application uses its own token, they all have slightly different installation commands. Besides different token, you will also use different addresses of Windows/Mac machines that host the monitored service.

Can SPM monitor metrics via http-remoting-jmx protocol (e.g. WildFly, JBoss, etc.)

Yes. The following steps are needed:

1. In /opt/spm/spm-monitor/conf/spm-monitor-config-YOUR_TOKEN-default.rt look for a line similar to this (the file name might be a bit different, depending on app type you have chosen):

   Code Block
   SPM_MONITOR_JAR="/opt/spm/spm-monitor/lib/spm-monitor-jvm.jar"

   Replace this line with something like the following (e.g. in case of WildFly adjust the path depending on where your WildFly is installed, like /opt/wildfly/bin/client/jboss-cli-client.jar):

   Code Block
   SPM_MONITOR_JAR="/opt/spm/spm-monitor/lib/spm-monitor-jvm.jar:/path/to/your/jboss-cli-client.jar"

   
   

2. Change the value of SPM_MONITOR_JMX_PARAMS in /opt/spm/spm-monitor/conf/spm-monitor-config-YOUR_TOKEN-default.properties. Of course, you can append to that additional JMX parameters, for example with password file location etc:

   Code Block
   SPM_MONITOR_JMX_PARAMS="-Dspm.remote.jmx.url=service:jmx:http-remoting-jmx://localhost:9990"

   
   

3. Restart spm-monitor with: sudo service spm-monitor restart

At this point the metrics will start appearing in charts. If they don't, run the diagnostics script (sudo bash /opt/spm/bin/spm-client-diagnostics.sh) to get fresh output of errors. If you see errors similar to:

Caused by: javax.security.sasl.SaslException: Authentication failed: all available authentication mechanisms failed:

java.io.FileNotFoundException: /opt/wildfly/standalone/tmp/auth/local8363680782928117445.challenge (Permission denied)

javax.security.sasl.SaslException: DIGEST-MD5: Cannot perform callback to acquire realm, authentication ID or password [Caused by javax.security.auth.callback.UnsupportedCallbackException

When should I run Standalone and when Embedded SPM monitor

A: Standalone SPM monitor runs as a separate process, while Embedded monitor runs embedded in the Java/JVM process.  Thus, if you are monitoring a non-Java application, Standalone monitor is the only option.  Standalone monitor is a bit more complex to set up when one uses it to monitor Java applications because it typically requires one to enable out-of-process JMX access, as described on Standalone SPM monitor page.  With Embedded monitor this is not needed, but one needs to add the SPM agent to the Java command-line and restart the process of the monitored application.  When running Standalone monitor one can update the SPM monitor without restarting the Java process being monitored, while a restart is needed when Embedded SPM monitor is being used.  To be able to trace transactions or database operations you need to use the Embedded SPM monitor.

Can I use SPM for (business) transaction tracing

A: Yes, see Transaction Tracing.

Can I move SPM client to a different directory

A: Yes. Starting with version 1.22.25, SPM client provides two scripts that can be used to move SPM client files/directories to another location:

1) Soft move script - Moves all SPM files/directories to a new location, but symlinks /opt/spm to the new location. Use this script if you are OK with having /opt/spm symlinked. This script is recommended for most situations since it keeps your SPM client installation completely in line with standard setup (all standard SPM client commands and arguments are still valid).

It accepts 1 parameter: new directory where SPM client should be moved to (if such directory doesn't exist, it will be created)

sudo bash /opt/spm/bin/move-spm-home-dir-soft.sh /mnt/some_dir

And that is it.

2) Full move script - Moves all SPM files/directories to a new location without leaving any SPM artifacts in /opt directory. Please note that using this script will make management of SPM client a bit harder (for example, there will be some required actions before running SPM client upgrade; also, if you are using in-process (javaagent) type of monitor you will need to adjust some paths in java arguments - more about that below).

It accepts 2 parameters: the current location (by default /opt/spm), and the new location (if such directory doesn't exist, it will be created). It can be run as:

sudo bash /opt/spm/bin/move-spm-home-dir.sh /opt/spm /mnt/some_other_dir/spm

If you are using in-process (javaagent) versions of SPM monitor to monitor Java-based applications, please adjust agent path(s) you use when starting application(s) you monitor with the SPM agent and make sure to use new location where old location was used before. Also add the following argument to the monitored application's Java process (you can add it right before -javaagent part; adjust the value to match directory you used):

-Dspm.home=/mnt/some_other_dir/spm

Note: When doing SPM client update on systems where SPM client was moved to different directory using "2) Full move script", you will first have to move SPM back to original /opt/spm directory:

sudo bash /opt/spm/bin/move-spm-home-dir.sh /mnt/some_other_dir/spm /opt/spm

and then proceed with regular update. After update is done, you can again move SPM client to location which suits you.

Is there an HTTP API I could use

A: Yes, see SPM API Reference.

I have multiple SPM applications installed on my machine, can I uninstall just one of them

A: Yes, you can use the following command for that (it accepts only one parameter, token of SPM application you want to uninstall):

sudo bash /opt/spm/bin/spm-remove-application.sh 11111111-1111-1111-1111-111111111111

Can I disable SPM agent without uninstalling the SPM client

A: Yes, just find its .properties file in /opt/spm/spm-monitor/conf and add to it:

SPM_MONITOR_ENABLED=false

After that restart the monitor to apply the change. In case of standalone agent, run:

sudo service spm-monitor restart

And in case of in-process agent, just restart the service that has this monitor's javaagent parameter.

How can I share my apps with other users

A: There are two options: Account Sharing or App Sharing. With Account Sharing, you invite others to your whole account, so they get access to all your applications, dashboards, notification hooks, alert rules, etc.,.  They can also create new apps under your account and invite other users. Depending on the role you assign to invitees, they may be able to administer your apps, dashboards, users or even billing info (change app plan, assign credit card, etc). Account Sharing is very convenient because as soon as a new app (SPM or Logsene or ...) is added to your account or new dashboard is created, all users added to your account get access to it.  Of course, the level of access depends on the role you initially assigned to each person.

Unlike Account Sharing, App Sharing is restricted only to a particular app. Nothing besides the shared app is accessible to the invited user (for example, dashboards are at the account level can can thus be shared only through Account Sharing). This option is useful if you want to be very restrictive about which apps can be seen by others or what kind of effect those others have on your team. With plain App Sharing guest can't see or edit alert rules created by your team, they can't use your team's notification hooks, etc.

Account Sharing and App Sharing is not exclusive.  You can use both of these two sharing types at the same time. You could Share Account with some users, and use App Sharing to share specific apps with other users.

To Share Account with other users, go to https://apps.sematext.com/ui/team/accounts
To Share App, go to https://apps.sematext.com/ui/team/apps

What is the difference between OWNER, ADMIN, BILLING_ADMIN, and USER roles

A: There are 3 common roles available when Sharing Account and Sharing App (OWNER, ADMIN, USER), and one role which is specific only to Account Sharing (BILLING_ADMIN).

Each account has one OWNER (if you created some account, you are its OWNER). Each app also has one OWNER (The OWNER of an app is OWNER of account under which some app was created. If you create an app under your account, you are the OWNER of that app. If some user with whom you've shared your account creates a new app under your account, you are again the OWNER of that app.  However, if that user creates a new app under his own account, he will be its OWNER).

Each account and app can have 0 or more ADMINS and USERS. If you added some user as ADMIN to your account, he also automatically gets the ADMIN role for all your apps (account role is transitive to app role).

ADMIN users can modify everything under your account/app except billing related info. They can: create/delete/update all dashboards/alerts/subscriptions/users... Users with USER role have read rights on everything except billing info (they can view all reports, dashboards, alerts...). They can even create/edit their own alerts and subscriptions on apps from shared account (but can't edit other user's alerts/subscriptions, only ADMINs can do that). If they were added to an Account (not to an App), they can also create their own dashboards and add other USERs to your account.

There is a special role available when Sharing Account - BILLING_ADMIN. This role has all rights as the standard ADMIN, but can also access/edit billing-related info. The only thing this role cannot do is change password of your account.

When would I want to add someone as BILLING_ADMIN

A: When you don't have a credit card that should be used for charging, but some other person has it, you should invite this person and give them the BILLING_ADMIN role. Similarly, if you created an account and defined a credit card, but now want somebody else to take care of all billing related activity (assigning plans and credit cards to various apps), you'd give them the BILLING_ADMIN role.

What might a typical use of roles for an organization with many employees look like

A: Typically you might have one person create an account via https://apps.sematext.com/users-web/register.doui/registration. This account might be considered a "parent" account for your whole organization.
Since the person who created the account would be its OWNER, this person is typically (but not necessarily) a team leader or manager, or somebody whose responsibility is to oversee servers/operations, typically in production. This person might then choose to share his/her whole Account with every other person from the team/organization to allow others to easily access all apps created under that account.

Some of the invited users might be given the ADMIN role, which gives them read and write (and invite) rights. Other invitees might get the USER role, which gives mostly just read rights (plus ability to create/edit their own dashboards/alerts/subscriptions which are available to everyone under the shared account). In some cases, account OWNER will not be able to handle billing related info and will want to invite 1 or more BILLING_ADMINs who will be able to define/edit/delete credit cards and choose plans to be used for apps under your account.

Agent Setup Automation

Is there an Ansible Playbook for the SPM client

A: Yes, see the Install and Configure playbooks, with examples.

Is there a Puppet Module for the SPM client

A: Yes, see the Install and Configure module, with examples.

Is there a Chef Recipe for the SPM client

A: Yes, see SPM client Chef Recipe example.

I am using SPM for Solr and I don't see any data only in Solr Components or Errors reports, what should I do

A: Most likely you are using standalone variant of Solr monitor. In that case, SPM monitor can't collect metrics which are available only when running in-process. If so, switch to in-process (javaagent) version of SPM monitor.