Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I can't get any metric #117

Open
misago162 opened this issue Jul 28, 2022 · 2 comments
Open

I can't get any metric #117

misago162 opened this issue Jul 28, 2022 · 2 comments

Comments

@misago162
Copy link

misago162 commented Jul 28, 2022

Hello

I have been tested this exporter in a machine with an openstack and in this machine, this exporter has worked fine. This openstack is a test environment with only one computer.

Then, I have installed in our production openstack environment without success, and I'm not able to see the problem. I can't get any metric. In this openstack environment we have 3 controller servers:

controller2: manage the network
controller3: manage the storage
controller1: manage the rest of the openstack components

I have installed the exporter in the controller1 with a dedicated openstack user to avoid use the admin user.

If from the own controller1 machine I try to get the metric I can see an 500 error:

# curl -v http://localhost:9183/metrics
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 9183 (#0)
> GET /metrics HTTP/1.1
> Host: localhost:9183
> User-Agent: curl/7.61.1
> Accept: */*
> 
* HTTP 1.0, assume close after body
< HTTP/1.0 500 Internal Server Error
< Server: BaseHTTP/0.6 Python/3.6.8
< Date: Thu, 28 Jul 2022 11:26:11 GMT
< 
* Closing connection 0
[root@nccontroller1 log]# 

This is the content of the prometheus-openstack-exporter.yaml file (although we have swift, it is commented just as a test to check if it was the problem):

[root@controller1 ~]# cat /etc/prometheus-openstack-exporter/prometheus-openstack-exporter.yaml 
# Example configuration file for prometheus-openstack-exporter
# Copyright (C) 2016-2019 Canonical, Ltd.
#

listen_port: 9183
cache_refresh_interval: 300  # In seconds
cache_file: /var/cache/prometheus-openstack-exporter/mycloud
cloud: mycloud
openstack_allocation_ratio_vcpu: 2.5
openstack_allocation_ratio_ram: 1.1
openstack_allocation_ratio_disk: 1.0
log_level: DEBUG

# Configure the enabled collectors here.  Note that the Swift account
# collector in particular has special requirements.
enabled_collectors:
  - cinder
  - neutron
  - nova
###  - swift
###  - swift-account-usage

# To export hypervisor_schedulable_instances metric set desired instance size
schedulable_instance_size:
    ram_mbs: 4096
    vcpu: 2
    disk_gbs: 20

# Uncomment if the cloud doesn't provide cinder / nova volumes:
#use_nova_volumes: False

## Swift

# There is no way to retrieve them using OpenStack APIs
# For clouds deployed without swift, remove this part
###swift_hosts:
###    - swift.xxx.es 

###    - export1 172.16.4.225:8080
###    - export2 172.16.4.226:8080
###    - export3 172.16.4.227:8080
###    - export4 172.16.4.228:8080

# There is no API to ask Swift for a list of accounts it knows about.
# Even if there were, Swift (in common case of Keystone auth, at
# least) only knows them by the corresponding tenant ID, which would
# be a less than useful label without post-processing.  The following
# should point to a file containing one line per tenant, with the
# tenant name first, then whitespace, followed by the tenant ID.
keystone_tenants_map:

# The reseller prefix is typically used by the Swift middleware to
# keep accounts with different providers separate.  We would ideally
# look this up dynamically from the Swift configuration.
# The Keystone middlware defaults to the following value.
reseller_prefix: AUTH_

ring_path: /etc/swift

# These will typically be read from /etc/swift/swift.conf.  If that
# file cannot be opened, then the Swift library will log an error and
# try to exit.  To run p-s-a-e as a user other than Swift, these
# settings must be set to the same values as Swift itself, and the
# above must point to an always-current readable copy of the rings.

hash_path_prefix:
hash_path_suffix:

[root@controller1 ~]# 

This is the content of the admin.novarc file

[root@controller1 ~]# cat /etc/prometheus-openstack-exporter/admin.novarc 
export OS_PROJECT_DOMAIN_NAME=Default
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_NAME=admin
#export OS_TENANT_NAME=admin
export OS_USERNAME=prometheus-exporter
export OS_PASSWORD=XXXXXXXX
export OS_AUTH_URL=https://xxxxx.xxx.xx:5000/v3
#export OS_INTERFACE=public
export OS_IDENTITY_API_VERSION=3
export OS_REGION_NAME=RegionOne
[root@controller1 ~]# 

And this is the log info when the service is running, nothing happens when I try to get the metrics. Note that in the development openstack, we get the same problem with the list_projects but it works fine, we can get the metrics.

Jul 28 13:28:51 nccontroller1 systemd[1]: Started prometheus-openstack-exporter.
Jul 28 13:28:53 nccontroller1 python3[32042]: Starting data gather thread
Jul 28 13:28:53 nccontroller1 python3[32042]: Client setup done, keystone ver 3
Jul 28 13:28:53 nccontroller1 python3[32042]: Error getting tenants.list, continue with projects.list
Jul 28 13:28:53 nccontroller1 python3[32042]: Error getting stats: Traceback (most recent call last):
                                                File "/opt/prometheus-openstack-exporter/prometheus-openstack-exporter", line 186, in _get_keystone_info
                                                  info["tenants"] = [x._info for x in keystone.tenants.list()]
                                                File "/usr/lib/python3.6/site-packages/keystoneclient/httpclient.py", line 893, in __getattr__
                                                  raise AttributeError(_("Unknown Attribute: %s") % name)
                                              AttributeError: Unknown Attribute: tenants
                                              
                                              During handling of the above exception, another exception occurred:
                                              
                                              Traceback (most recent call last):
                                                File "/opt/prometheus-openstack-exporter/prometheus-openstack-exporter", line 262, in run
                                                  prodstack.update(self._get_keystone_info(keystone))
                                                File "/opt/prometheus-openstack-exporter/prometheus-openstack-exporter", line 189, in _get_keystone_info
                                                  info["tenants"] = [x._info for x in keystone.projects.list()]
                                                File "/usr/lib/python3.6/site-packages/keystoneclient/v3/projects.py", line 142, in list
                                                  **kwargs)
                                                File "/usr/lib/python3.6/site-packages/keystoneclient/base.py", line 86, in func
                                                  return f(*args, **new_kwargs)
                                                File "/usr/lib/python3.6/site-packages/keystoneclient/base.py", line 448, in list
                                                  list_resp = self._list(url_query, self.collection_key)
                                                File "/usr/lib/python3.6/site-packages/keystoneclient/base.py", line 141, in _list
                                                  resp, body = self.client.get(url, **kwargs)
                                                File "/usr/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 386, in get
                                                  return self.request(url, 'GET', **kwargs)
                                                File "/usr/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 545, in request
                                                  resp = super(LegacyJsonAdapter, self).request(*args, **kwargs)
                                                File "/usr/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 248, in request
                                                  return self.session.request(url, method, **kwargs)
                                                File "/usr/lib/python3.6/site-packages/keystoneauth1/session.py", line 943, in request
                                                  raise exceptions.from_response(resp, method, url)
                                              keystoneauth1.exceptions.http.Forbidden: You are not authorized to perform the requested action: identity:list_projects. (HTTP 403) (Request-ID: req-c1121018-2efb-4768-bddb-cfdd6558822f)

Tell me if you need I upload more information.

Any help to find the problem would be appreciated. Thanks a lot.

@misago162
Copy link
Author

Hello, any advice or idea about where is the problem? Thanks a lot

@jagter
Copy link

jagter commented May 9, 2023

The OpenStack user prometheus-exporter doesn't have the permissions to do a list of projects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants