About this App

Motivation:
SQL Azure is a ‘database as a service’ offering from Microsoft. When we look at it from a conventional DBA’s perspective (A DBA who is used to manage on premise SQL instances), one of the important things he/she’s going to miss is “A comprehensive monitoring tool” which allows him/her to live monitor, mitigate and prevent any unwanted situations/anomalies and keep track of his databases’ performance.

Architecture of the application:

clip_image001

WCF service:
For either windows store or windows phone apps, since we cannot use the system.data namespace we introduced a layer in between: a WCF web service. The authentication of the users, server names and getting all the necessary monitoring details, in fact almost all of the business logic is going to run here. To have this service working Azure admin have to set the firewall to allow access to applications on windows azure.

End User application:
Users give input of their credentials, once they are authenticated user can click around the app to fetch the monitoring details.

How we implemented this:

We made an app which can be run on any windows device (except phones for now) and allow DBAs/users to monitor the health of his SQL Azure instances/databases. He can monitor on the go, from home, restaurant, from anywhere – Just a mobile device and internet connectivity is all that is needed.

 

This app is developed by:

Akbar Farishta

Varma Gadhiraju

Chandra Sekhar Maddila 

Any feedback, support related queries?  Write us @ feedback.asm2015@outlook.com

Setting up Logins to monitor Azure SQL Databases

Hello Everyone!

In this post we will discuss, “How to setup Logins with minimal permissions to monitor Azure SQL Databases?”

Recently, I and other teammates were brain storming the idea of writing a monitoring application for Azure databases and that is when we came across this question. We wanted to make sure the admins can use a login with very few permissions while logging into our application to monitor the databases, we wanted them to feel secure and trust that the application will not have capability to change anything in the databases, in fact not even be able read the data from the databases.

Let us get to work!

Step1: Create a login

Login to the server instance using your admin account.

–Create a login with password

Create LOGIN APPLOGIN with password=’abcd123?’

If you attempt to access the instance with the above login you will get the below error

******

Cannot open database “master” requested by the login. The login failed.

Login failed for user ‘APPLOGIN’.

This session has been assigned a tracing ID of ‘<>’. Provide this tracing ID to customer support when you need assistance. (Microsoft SQL Server, Error: 4060)

******

To be able to login to the server we need to add this login to the master database as user, with DEFAULT_SCHEMA = GUEST

–Create user under master database

CREATE USER [APPUSER] FOR LOGIN [APPLOGIN] WITH DEFAULT_SCHEMA=[guest]

GO

When we login using APPLOGIN into management studio, we will able to see the databases listed, however we will not be able to view any of the user tables in the user databases.

Step 2: Setup a database

Let us create a database to monitor. We have to use the admin login as we cannot create a database using the APPLOGIN

–Create TEST_DB

CREATE DATABASE TEST_DB

GO

Switch to TEST_DB database

–Create a simple table under TEST_DB, we will use this to test permissions.

CREATE TABLE TEST_TABLE

(

c1 int NOT NULL

CONSTRAINT PK_sample_table PRIMARY KEY (c1)

)

GO

Step 3: Add user and permissions

To be able to run any queries against TEST_DB database using APPLOGIN we need to add it as a user.

–Add login as user under TEST_DB

CREATE USER [APPUSER] FOR LOGIN [APPLOGIN] WITH DEFAULT_SCHEMA=[guest]

GO

To monitor any database we need “VIEW DATABASE STATE” and “VIEW DEFINITION” Permissions for the user account

–Provide “VIEW DATABASE STATE” permission to the user [APPUSER]

GRANT VIEW DATABASE STATE TO APPUSER

–Provide “VIEW DEFINITION” permission to the user [APPUSER]

GRANT VIEW DEFINITION TO APPUSER

Step 4: Verify permissions

Now that we have the Login, user, and database to monitor in place, let’s verify if the user is able to select any tables in the TEST_DB, because our main aim is to make sure the monitoring LOGIN has minimal permissions

Login to the server using APPLOGIN and take a connection with TEST_DB database and run the below query.

select * from TEST_TABLE

Boom! We get the below error

Msg 229, Level 14, State 5, Line 1

The SELECT permission was denied on the object ‘TEST_TABLE’, database ‘TEST_DB’, schema ‘dbo’.

You can run other queries like, create table, update a row etc. to make sure there the permissions are at minimum.

Step 5: Monitor the database

We will be using the queries in below article to monitor Azure SQL databases

http://msdn.microsoft.com/library/azure/ff394114.aspx

Make sure you are connected to TEST_DB using APPLOGIN (monitoring login)

–To monitor database size

SELECT SUM(reserved_page_count)*8.0/1024

FROM sys.dm_db_partition_stats;

GO

SELECT SUM(reserved_page_count)*8.0/1024

FROM sys.dm_db_partition_stats;

GO

— Find top 5 queries

SELECT TOP 5 query_stats.query_hash AS “Query Hash”,

SUM(query_stats.total_worker_time) / SUM(query_stats.execution_count) AS “Avg CPU Time”,

MIN(query_stats.statement_text) AS “Statement Text”

FROM

(SELECT QS.*,

SUBSTRING(ST.text, (QS.statement_start_offset/2) + 1,

((CASE statement_end_offset

WHEN -1 THEN DATALENGTH(st.text)

ELSE QS.statement_end_offset END

– QS.statement_start_offset)/2) + 1) AS statement_text

FROM sys.dm_exec_query_stats AS QS

CROSS APPLY sys.dm_exec_sql_text(QS.sql_handle) as ST) as query_stats

GROUP BY query_stats.query_hash

ORDER BY 2 DESC;

GO

Finally, we have a login created with minimal permission to monitor the Azure SQL databases. Using this login there is no risk of any data read or data modification.

How do i make sense out of the graphs and the data? – “Basic Details Tab”

Connection count graph

The connection count graph provides information about the number of connections that are coming into your Azure SQL database. It is an auto refreshed graph with a refresh interval of 15 seconds. we’re just taking count of rows from sys.dm_exec_connections DMV and plotting them in a neat timeline graph.
More details about connections are displayed under tab#3: “Connections Overview”

Total size Vs used chart

The pie chart you see on bottom left corner provides you information about the total size / max size of your database (based on the service tier you selected to create the database in Azure portal) and how much space is consumed by the existing objects. DBAs can leverage this information to understand how much free space is left in their databases and does it need any expansion (you can change the tier from your azure portal if you need to expand your database ).

Total/Max size is shown in GBs and used space is shown in MBs

More info on service tiers: https://msdn.microsoft.com/en-us/library/azure/dn741336.aspx

Top 5 tables chart

We’re plotting a pie chart of top 5 large tables or top 5 consumers of disk space in your Azure SQL database. Tables’ respective sizes are shown in MBs

How do i make sense out of the graphs and the data? – “Resource Usage Tab”

The data shown in this tab (“Resource usage”) is pulled from a DMV sys.dm_db_resource_stats

It returns CPU, I/O, and memory consumption for an Azure SQL Database database. One row exists for every 15 seconds, even if there is no activity in the database. Historical data is maintained for one hour.

clip_image001

Average memory utilization in percentage of the limit of the service tier.

So, if you see the percentage CPU utilization is 7% that means 7% of CPU is being utilized out of the total CPU computational resources allocated to your Azure SQL database based on the service tier of the database.

For more info: https://msdn.microsoft.com/en-IN/library/dn800981.aspx

How do i make sense out of the graphs and the data? – “Connections Overview Tab”

The data shown in this tab (“Connections Overview”) is pulled from DMV sys.database_connection_stats

It contains statistics for SQL Database database connectivity events, providing an overview of database connection successes and failures.

image

 

For more info: https://msdn.microsoft.com/en-IN/library/dn269986.aspx

For Info on various connection events (failure, terminated, throttled etc…) and possible causes, refer to the link below:

https://msdn.microsoft.com/en-IN/library/dn270018.aspx#eventtypes

For comprehensive understanding of the root causes for various connection failure, throttling and termination events please look out for connection specific events in the link above.

How do i make sense out of the graphs and the data? – “Top CPU Queries Tab”

The data shown in this tab (“Top CPU Consumers”) is pulled from DMV sys.dm_exec_query_stats joined with query_text.

It returns aggregate performance statistics for cached query plans in SQL Server. The view contains one row per query statement within the cached plan, and the lifetime of the rows are tied to the plan itself. When a plan is removed from the cache, the corresponding rows are eliminated from this view.

execution_count

Number of times that the plan has been executed since it was last compiled.

total_elapsed_time

Total elapsed time, reported in microseconds (but only accurate to milliseconds), for completed executions of this plan.

total_worker_time

Total amount of CPU time, reported in microseconds (but only accurate to milliseconds), that was consumed by executions of this plan since it was compiled.

For natively compiled stored procedures, total_worker_time may not be accurate if many executions take less than 1 millisecond.

  • Percentage of CPU time is nothing but the percentage of total_worker_time*100/total_elapsed_time

This metric is very useful to understand which statements spends lot of time on CPU. May be due to bad stats, execution plans, heavy recompilations etc…

  • Average CPU time is total_worker_time/execution_count

So, if a statement runs for 5 times and spends 100ms on CPU then average CPU time reported will be 100/5 = 20ms. This metric is very useful to understand which statements doesn’t comes into the system quite frequently but consumes lot of CPU time per execution.

Privacy Policy

This App does not store any user credentials or make any modifications to the SQL server databases. We use the credentials(we prefer READONLY credentials, in fact) to login to the SQL server and fetch the monitoring information for that instance from various DMVs.

We don’t do any data modifications as well as we don’t access any user data that is stored in ‘user created tables’.

The App is enabled with SSL encryption for data transfer between app and the Webservice.