Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 107 Next »


Getting Started


What should I do if I can't find the answer to my question in the FAQ?
A: You can reach us at spm-support@sematext.com or tweet @sematext.

Is there an On Premise version of SPM that I can run on my own servers?

A: Yes there is.  Please contact us if you need more information or want to get a 30-day trial copy of On Premise SPM edition.

Is there a limit to how many servers I can monitor with SPM?

A: Each plan has a limit, but it is pretty high and if you need to monitor more servers just let us know.

Can I use SPM to monitor any application?

A: SPM can be used to monitor any UNIX server and any Java application in addition to monitoring specific applications.

Does SPM work on Windows?

A: We have not tested it, but it should work, except for the installer script and a cron job for log management. It should work completely under Cygwin.

Can I run the complete SPM on my own servers?

A: Yes, SPM on Premises is available. Please get in touch with us.

Which Applications can SPM monitor?

A: SPM can monitor Hadoop, HBase, Cassandra, ElasticSearch, Solr/SolrCloud, MySQL, MariaDB, Redis, Memcached, ZooKeeper, Kafka, Storm, Spark, AWS/CloudWatch, SenseiDB, JVM, etc.  You can also feed it Custom Metrics.

How do I install Collectd?

A: Please see our Collectd Installation instructions. Note that if you have previously installed SPM client while Collectd wasn't installed yet or wasn't running on your system, and now you want to install Collectd to start monitoring OS related stats, you will have to re-run SPM client installer after Collectd is installed and started.

How do I configure Collectd for SPM if I am already using it?

A: SPM requires the Collectd "csv" plugin to be enabled and configured like this:

<Plugin csv>
  DataDir /opt/spm/collectd/logs
  StoreRates true
</Plugin>

 If you are already using Collectd and if the DataDir is already pointed to some other directory, simply symlink the /spm/collectd/logs directory to the directory specified in DataDir, like this:

sudo mkdir -p /opt/spm/collectd  # if needed
sudo ln -s /some/existing/collectd/data/dir /spm/collectd/logs
You will also have to merge the remaining part of the Collectd config required by SPM.

Is there a Chef Recipe for the SPM client?

A: Yes, see SPM client Chef Recipe example.

What is the difference between OWNER, ADMIN, and USER roles?

A: OWNER owns the whole account and is the only role that can see and change credit card information and plans.  There is only one OWNER for each SPM App.  ADMIN users can modify everything in an SPM App except parts that have to do with billing.  Users with USER role have mostly read-only rights, though they can create and modify their own Dashboards.

Can hostnames in SPM UI be obfuscated or customized?

A: Yes, you can obfuscate or alias hostnames starting with SPM client version 1.17.7. This feature allows you to:
  • never send your real hostnames over the Internet
  • use custom hostname in SPM UI in case original hostname is too cryptic (e.g. have SPM shows "my-solr-host1" instead of "ip-12-123-321-123")

 

To achieve this, after SPM client is installed, open /opt/spm/properties/spm-sender.properties file and just add desired hostname to the value of property spm_sender_hostname_alias.

After that, restart SPM sender with sudo /etc/init.d/spm-sender restart

Few notes to keep in mind:

  • you can do this change at any point, even after SPM client was running on your machine for the past few months or years - you just have to upgrade SPM client to version 1.17.7 or greater
  • old data will still be seen in SPM under the old hostname, while new data (after hostname change) will be displayed under the new hostname
  • if you are installing SPM client on some machine for the first time, and you want to be 100% sure its original hostname never leaves your network, define your hostname alias in spm-sender.properties file immediately after you complete "1. Package installation" step and before you begin with "2. Client configuration setup" step described on page http://apps.sematext.com/spm-reports/client.do.

Can I reduce the amount of logs generated by SPM monitor?

Yes, starting with 1.20.6 version of SPM client you can make SPM monitor reduce the number of messages it logs every minute (otherwise it logs detailed info about its monitoring lifecycle). To do that, change properties files of your SPM applications. You can find those files with:

ls /opt/spm/spm-monitor/conf/spm-monitor-config-*.properties

You can adjust one or more of them, depending on which application's monitor log output you want to reduce. At the bottom of those files add the following line:

SPM_MONITOR_LOGGING_LEVEL=reduced

After you have made the changes, restart your SPM monitor – if you are using standalone variant, run: sudo /etc/init.d/spm-monitor restart , otherwise restart your application/java process.

Can I use SPM to monitor multiple applications running on the same server / VM?

A: Yes. There are really 2 different scenarios here:

1) If each of those applications should be monitored under a different SPM application (e.g., you could have Solr running on your server along with some Java app and you want to monitor both - Solr would be monitored with SPM application of Solr or SolrCloud type, while the Java app would be monitored with SPM application of JVM type), just complete all installation steps from http://apps.sematext.com/spm-reports/client.do for each of them separately.

2) If you want them monitored under the same SPM application (e.g., you have 3 Solr instances running on a server), you must use different JVM name for each of them. To do this, "1. Package installation" step on http://apps.sematext.com/spm-reports/client.do should be run only once on this machine, while "2. Client configuration setup" step should be run once for each of the 3 Solr instances. When running script /opt/spm/bin/spm-client-setup-conf.sh in step "2. Client configuration setup", you should add jvmname parameter (and value) at the end of parameter list, like this:

sudo bash /opt/spm/bin/spm-client-setup-conf.sh 11111111-1111-1111-1111-111111111111 solr javaagent jvmname:solr1

In this example, we are setting up things for 3 separate Solr processes, monitored under JVM names: solr1, solr2 and solr3 (you choose any names), so this is adjusted command for solr1 instance.

In the remaining sub-steps of "2. Client configuration setup", replace word "default" with the jvmname value you just used.

"2. Client configuration setup" step will have to be repeated N times, once for each monitored application (in our example 3 times with 3 different jvmname values).

Note: By using this kind of setup, you will be able to see JVM stats of all 3 processes separately (JVM name filter is used to do the filtering). When it comes to other, non-JVM stats, they will be aggregated into single value (for instance, request rate chart will show sum of request rate value of these 3 Solr instances). If you want to see even non-JVM stats separately, you will have to create 3 separate SPM Solr applications (one for each Solr instance running on this machine).

Can I use SPM to monitor my service which runs on Windows or Mac OS X?

SPM client installers currently exist only for Linux, however there is still a way to monitor your service. If you are OK with installing SPM client on separate linux box, and pointing it (more about that further below) to your service (which should be monitored) running on Windows or Mac, you can use SPM to monitor all non-OS related metrics. That means you will not be able to see CPU or system Memory metrics, but metrics specific to your process will be displayed (for example, in case of Solr, you will see all search, index, cache metrics along with all JVM metrics like pool memory, GC stats, JVM threads etc; as another example, in case of MySQL you will see all metrics related to connections, users, queries, handlers, commands, MyISAM, InnoDB, MySQL traffic, etc) .

When monitoring Solr, Elasticsearch, HBase, Hadoop and other Java-based services, you will have an option to choose between using In-process (javaagent) or Standalone monitor. The workaround described here requires the use of standalone monitor variant. Here's what you'd need to do to see your metrics in SPM:

  1. Install spm-client on any Linux box (you can use this box for anything else, it is needed here just to run a process which collects metrics and sends them to Sematext), following steps from http://apps.sematext.com/spm-reports/client.do . In step 1 choose the "Other" tab to create a minimal installation.  This will not install modules needed for monitoring OS metrics - if those modules were installed, they would collect OS metrics of your Linux box, which is probably something you don't need. If you decide to follow instructions from some other tab, keep in mind OS metrics displayed in SPM UI will not be OS metrics of your Windows/Mac machine.
  2. In step 2, if you are given a choice between In-process and Standalone monitor, choose Standalone monitor. It will use remote JMX to collect metrics from your Windows/Mac machine. Just follow instructions given on Standalone tab.  The only difference you will want to make is in -Dspm.remote.jmx.url parameter used in monitor properties.  You will know where to adjust it if you follow standard instructions for Standalone monitor. While this parameter will usually have a value like -Dspm.remote.jmx.url=localhost:3000, in this case you will have to replace localhost with the address of your Windows/Mac machine(s) by which it can be reached from your helper Linux box.
  3. If you are monitoring something that doesn't offer a choice between In-process and Standalone monitor, installation instructions at http://apps.sematext.com/spm-reports/client.do will explain where you can define your own address. Instructions typically use localhost, but you can instead use something like 10.1.2.3 or my-win-solr-server.mycompany.com to point to the machine that hosts the service you intend to monitor.

In case you want to monitor multiple machines belonging to the same cluster in this way, you can still use SPM installed on a single Linux helper box. Just do this:

  • Create a separate SPM application in SPM UI (https://apps.sematext.com/spm-reports/registerApplication.do) for each of the machines which should be monitored
  • For each of those SPM applications (and your monitored machines), go through installation process described in http://apps.sematext.com/spm-reports/client.do . Note: Since each SPM application uses its own token, they all have slightly different installation commands. Besides different token, you will also use different addresses of Windows/Mac machines that host the monitored service.

When should I run Standalone and when Embedded SPM monitor?

Standalone SPM monitor runs as a separate process, while Embedded monitor runs embedded in the Java/JVM process.  Thus, if you are monitoring a non-Java application, Standalone monitor is the only option.  Standalone monitor is a bit more complex to set up when one uses it to monitor Java applications because it typically requires one to enable out-of-process JMX access, as described on Standalone SPM monitor page.  With Embedded monitor this is not needed, but one needs to add the SPM agent to the Java command-line and restart the process of the monitored application.  When running Standalone monitor one can update the SPM monitor without restarting the Java process being monitored, while a restart is needed when Embedded SPM monitor is being used.  At Sematext we use Embedded monitor in almost all cases and use Standalone mode only for monitoring non-Java applications like MySQL and Apache. 

Can I move SPM client to a different directory?

Yes. Starting with version 1.22.3, SPM client provides a script one can use to move SPM client files/directories to another location. It accepts 2 parameters: the current location (by default /opt/spm), and the new location (if such directory doesn't exist, it will be created). It can be run as:

sudo bash /opt/spm/bin/move-spm-home-dir.sh /opt/spm /mnt/some_other_dir/spm

If you are using in-process (javaagent) versions of SPM monitor to monitor Java-based applications, please adjust agent path(s) you use when starting application(s) you monitor with the SPM agent and make sure to use new location where old location was used before. Also add the following argument to the monitored application's Java process (you can add it right before -javaagent part; adjust the value to match directory you used):

-Dspm.home=/mnt/some_other_dir/spm

 

and restart your application/server.

 

Note: When doing SPM client update on systems where SPM client was moved to different directory, you will first have to move SPM back to original /opt/spm directory:

sudo bash /opt/spm/bin/move-spm-home-dir.sh /mnt/some_other_dir/spm /opt/spm

and then proceed with regular update. After update is done, you can again move SPM client to location which suits you.

Can I create new SPM applications programmatically via an API?

Yes, there is an HTTP API that let's you create apps programmatically. Please email spm-support@sematext.com to get the details – the details are not included here to discourage easy abuse.

General Troubleshooting

Can I install SPM client on servers that are behind a proxy?

A: Yes.  If you are installing the RPM, add this to /etc/yum.conf:

proxy=http or https://proxy-host-here:port
proxy_username=optional_proxy_username
proxy_password=optional_proxy_password

 

If you are using apt-get, set the http_proxy environmental variable:

export http_proxy=http://username:password@yourproxyaddress:proxyport

 

Can SPM client send data out from servers that are behind a proxy?

A: Yes, just provide the proxy info in the /opt/spm/properties/spm-sender.properties file:

 

spm_sender_proxy_host=
spm_sender_proxy_port=
spm_sender_proxy_user_name=
spm_sender_proxy_password=

 

When installing SPM client, I see "The certificate of `pub-repo.sematext.com' is not trusted" or similar error.  How can I avoid it?

A: There can be multiple reasons for this, most likely:

  • system time on your machine is not correct and adjusting it should fix the problem
  • you are installing SPM client as root user - in some cases that can cause this error and installing SPM client as a different user may help

If none of the above eliminates the problem for you try adding a flag to avoid certificate checking. In case the command with wget is failing in your case, add "--no-check-certificate" as wget argument. If command with curl is failing, add "-k" flag.

I see only my system metrics (e.g. CPU, Memory, Network, Disk...), but where is the rest of my data?

A: Make sure you have followed all steps listed on installation instructions page. Package installation steps should be done first, followed by Client configuration setup. If you have done that and you still don't see application metrics, run sudo bash /opt/spm/bin/spm-client-diagnostics.sh to generate diagnostics package and send it to spm-support@sematext.com with description of your problem.

I DO NOT see any system metrics (e.g. CPU, Memory, Network, Disk), what could be the problem?

A: SPM depends on Collectd to display system metrics. If SPM was showing your system metrics in the past and has now stopped, first check if Collectd is still installed and running. Also check suggestions from "I rebooted my server and now I don't see any data in my graphs. What should I check?" If SPM never showed system metrics for your application, it may be that Collectd wasn't installed or running at the time when you installed SPM client. In that case, ensure Collectd is installed and running and after that re-run SPM client installer for your application.

I am using SPM for Solr and I don't see any data on Solr and JVM reports, what is the problem?

You should probably enable JMX in your Solr. Add or uncomment the <jmx /> directive in solrconfig.xml and restart Solr.  See http://wiki.apache.org/solr/SolrJmx for more info.

I am using SPM for Solr and I don't see any data only in Solr Components or Errors reports, what should I do?

Most likely you are using standalone variant of Solr monitor. In that case, SPM monitor can't collect metrics which are available only when running in-process. If so, switch to in-process (javaagent) version of SPM monitor.

I am using SPM for Elasticsearch monitor and I don't see Index (and/or Refresh/Flush/Merge) stats, why is that?

SPM for Elasticsearch monitor collects Index stats only from primary shards, so it is possible that you installed SPM monitor on some Elasticsearch node which hosts only replicas. The same is also true for Refresh/Flush and Merge stats. Also note that SPM Elasticsearch monitor should be installed on all your Elasticsearch nodes to get the complete picture of your cluster in SPM Reports UI.

I registered for SPM more than 5 minutes ago and I don't see ANY of my data / I am getting errors when starting SPM Sender or Monitor, what should I check?

A: Here are a few things to check and do:
  1. Log into your monitored servers and make sure Collectd and SPM Sender are running.
  2. Check if system time is correct. If not, you should adjust the time, restart the SPM Sender with:
    sudo /etc/init.d/spm-sender restart

    and restart SPM Monitor by restarting your server which is being monitored (in case in-process/javaagent variant of SPM Monitor) or (in case of standalone SPM Monitor installation):

    sudo /etc/init.d/spm-monitor restart
  3. Make sure disks are not full.
  4. Make sure user spmsender can have more than 1024 files open:

    sudo vim /etc/security/limits.conf
    spmsender     -    nofile    32000
    
    sudo vim /etc/pam.d/su
    session    required   pam_limits.so

    Restart SPM Sender after the above changes.

  5. Check if hostname of your server is defined in /etc/hosts
  6. If you are starting your Jetty (or some other server) with command like "java ... -jar start.jar ..." and using inprocess (javaagent) version of monitor, make sure -D and -javaagent definitions occur before "-jar start.jar" part in your command
  7. If none of the suggestions helped, run sudo bash /opt/spm/bin/spm-client-diagnostics.sh to generate diagnostics package and send it to spm-support@sematext.com

My server stopped sending metrics, so why do I still see it under Hosts Filter?

A: Filters have 1 day granularity, which means that a server will be listed under Hosts filter until 24 hours since it last sent data have passed.  For example, if a server stopped sending data at 1 PM and if at 8 PM you are looking at the last 6 hours of data (for a period from 2 PM until 8 PM) you will not see data from this server on the graph, but you will still see this server listed under the Hosts filter until 1 PM on the following day.  After 1 PM on the following day this server should disappear from the Hosts filter.

I rebooted my server and now I don't see any data in my graphs. What should I check?

A: Here are a few things to check and do:
  1. Make sure Collectd is running: sudo service collectd restart. Make sure Collectd is set up to start at boot.
  2. Make sure SPM Sender is running: sudo /etc/init.d/spm-sender restart
  3. Make sure disk is not full: df -h
  4. Make sure maximal open files limit was not reached: see "I registered for SPM more than 5 minutes ago and I don't see any of my data, what should I check?"

I see a server is using swap, but the swap I/O graph is empty - how can that be?

A: You may be using an old version of Collectd. A bug with incorrect swap I/O values was fixed in Collectd 4.10. Make sure you are using Collectd 4.10 or newer. You can install/upgrade Collectd with yum, apt-get, etc., or you can download a version from http://collectd.org/download.shtml.

 

How come Disk Space Usage report shows more free disk space than df command?

A: SPM reports both free and reserved disk space as free, while df does not include reserved disk space by default.

 

I changed my server's hostname and now I don't see new data in my graphs. What should I do?

A: Simply restart SPM Sender:

 

sudo /etc/init.d/spm-sender restart

 

Can I specify which Java runtime the SPM client should use?

A: Yes, you can edit the /opt/spm/properties/java.properties file where you can specify the location of Java you want the SPM client (the SPM Sender, and SPM monitor if you are running the standalone version) to use.

Can SPM client use HTTP instead of HTTPS to send metrics from my servers?

A: Yes. Since version 1.22.0, SPM sender by default uses HTTPS to send metrics data to Sematext servers. If you prefer to use HTTP instead (for example, if you are running SPM on premises or if you don't need metric data to be encrypted when being sent to SPM over the Internet), you can adjust that in /opt/spm/properties/spm-sender.properties by changing protocol to http in property:

spm_sender_receiver_url=https://spm-receiver.sematext.com/receiver/v1

 

Solr Monitoring

How do I enable JMX in Solr?

A: Add or uncomment the <jmx/> directive in solrconfig.xml and restart Solr.  See http://wiki.apache.org/solr/SolrJmx for more info.

Elasticsearch Monitoring

Why doesn't the number of documents I see in SPM match the number of documents in my Elasticsearch index?

A: SPM collects index stats from primary shards only.  To see the total number of documents in an index, select all shards in that index and choose "sum".  The list of shards and the "sum" function can be found in the "Shard filter" in the Index Stats report.

Can SPM collect metrics even when Elasticsearch HTTP API is disabled?

A: Each SPM agent collects Elasticsearch metrics only from the local node by accessing the Stats API via HTTP.  To allow only local access add the following to elasticsearch.yml. Don't forget to restart each ES node to whose elasticsearch.yml you add this.
http.host: "127.0.0.1"

 

HBase Monitoring

How do I enable JMX in HBase?

A: Please see HBase Metrics page for instructions.

Do I need to add a separate SPM Application for each HBase server/node I want to monitor?

A: No, one Application is enough. Think of an SPM "Application" as a "HBase Cluster". Thus, to monitor N HBase servers that belong to the same cluster you would create just a single SPM Application and use its Token in SPM configuration file on all HBase servers that are a part of this cluster.

Why don't some HBase metrics graphs have any data?

A: There could be 2 possible reasons:

  1. Some metrics are for RegionServers (HBase slaves), some for HBase Master. Thus, if you select the Master node in the UI, graphs that contain Slave-specific metrics will be blank and vice-versa.
  2. Different versions of HBase provide different metrics. Thus, if you have an older version of HBase, it may not be providing all metrics that SPM collects and graphs.

Which versions of HBase does SPM support?

A: SPM has been tested with HBase 0.90 and 0.92, but will work for newer versions, including all CDH versions.

Kafka Monitoring

Why am I not seeing all Kafka metrics if I'm running a 0.8.x version of Kafka that is pre-0.8.2?

A: Kafka had great metrics in 0.7x versions and then in pre-0.8.2 the metrics had a lot of issues.  In short, the MBeans Kafka exposed via JMX were named poorly making them very difficult/impossible to parse systematically.  We worked with the Kafka developers to fix that in Kafka 0.8.2 (see https://issues.apache.org/jira/browse/KAFKA-1481).  To make your Kafka Producers, Consumers, and Brokers properly monitorable, you should upgrade to 0.8.2 (as soon as it is released) or later.

How can I see metrics for Kafka Producers and Brokers and Consumers?

A: To see metrics for Kafka Producers, Brokers, and Consumers make sure to run SPM client on all of them.  Each of them has its own metrics.  Kafka Brokers do not expose Producers' or Consumers' metrics, so ensure you have SPM client on all three Kafka tiers.

Can I see metrics for non-Java Kafka Producers and Consumers?

A: No, because none of them seem to expose metrics the way Kafka's own Producer and Consumer implementations expose metrics.
 

Upgrading

How do I upgrade the SPM client?

A: If you have previously installed the SPM client package (RPM, Deb, etc.) as described on https://apps.sematext.com/spm-reports/client.do simply upgrade via apt-get (Debian, Ubuntu, etc.), or yum (RedHat, CentOS, etc.), or zypper (SuSE):

sudo apt-get update && sudo apt-get install spm-client  # NOTE: this does not update the whole server, just spm-client
sudo yum clean all && sudo yum update spm-client        # NOTE: this does not update the whole server, just spm-client
sudo zypper up spm-client

After that is done, also do:

  • if you are using SPM monitor in in-process/javaagent mode - restart monitored server (restart your Solr, Elasticsearch, Hadoop node, HBase node... Exceptions: In case of Memcached, no need to restart anything; in case of Redis only standalone SPM monitor exists so check below how to restart it)

OR

  • if you are using standalone SPM monitor, restart it with:

sudo /etc/init.d/spm-monitor restart

 

If you used bash script based installer, then just run the installation command again ("curl -ko installer.sh ...." or "wget --no-check-certificate -O installer.sh ..."). There is no need to go through "Client configuration setup" steps from https://apps.sematext.com/spm-reports/client.do since the existing configuration will be preserved.

After that is done, next step is the same as with apt-get/yum/zipper upgrade - you should restart monitored server (if using in-process/javaagent monitor) or standalone SPM monitor.

 

Note: If you moved SPM client from /opt/spm to different directory, you will first have to move SPM back to original /opt/spm directory:

sudo bash /opt/spm/bin/move-spm-home-dir.sh /mnt/some_other_dir/spm /opt/spm

and then proceed with regular update. After update is done, you can again move SPM client to location which suits you.

Uninstalling

How do I uninstall the SPM client?

A: On servers where you want to uninstall the client do the following:

  1. remove spm-client, for instance: sudo apt-get purge spm-client   OR   sudo yum remove spm-client
  2. after that, ensure there are no old logs, configs, etc. by running the following command: sudo rm -R /opt/spm
  3. if you used in-process (javaagent) version of monitor, remove "-javaagent" definition from startup parameters of process which was monitored

 

Note: in case you used installer described on "Other" tab on http://apps.sematext.com/spm-reports/client.do , instead of commands from step 1 run: sudo bash /opt/spm/bin/spm-client-uninstall.sh . After that proceed with steps 2 and 3 described above.

Billing

Which credit cards are accepted?

A: We accept all major credit cards - Visa, MasterCard, American Express, JCB, Discover, and Diners Club.

Can I be invoiced instead of paying with a credit card?

A: Yes, we offer a pre-payment option.  When you select this option just email us and tell us how much you want to pre-pay.  We'll invoice you and once we receive your payment we'll enter in the system and you'll be good to go.  Every month when we do the billing we'll subtract your usage cost for the previous month from your balance in our system.  We'll notify you before your balance gets too low, so that we can repeat the invoicing process.

How often will I get billed?

A: We bill on a monthly basis and send an email to notify you of the amount.

Can the billing email be sent to our Accounts Payable/Accounting instead of me?

A: Yes, when you select your plan and payment method you will also be able to enter an alternative billing email address.

Do I have to commit or can I stop using SPM at any time?

A: There is no commitment and no contract. When/if you want to stop using SPM you simply stop sending us your metrics and/or switch to the Free plan.

 

  • No labels