Showing posts with label expert. Show all posts
Showing posts with label expert. Show all posts

Wednesday, March 21, 2012

Degrading performance. Expert help appreciated

Greetings,
In our current project, we use a combination of SQL Server 2000 and
ADO. Our database is very simple and small: just about 10 tables with
300K records at most (in just one of the tables). We need, however,
very fast responses to our queries. With that in mind, we designed and
optimized all queries and indices in such a way that each query takes
less that 20 milliseconds, as measured using the SQL profiler under a
normal load.
Each client application opens a single connection to the database and
the queries are funneled through that connection. Each query is a
individual transaction, i.e. it is fenced by Begin Tran...End Tran. We
use mostly stored procedures, which are executed via the _Command
object from ADO. In just a couple of cases, we use _Recordset.
Under a *stress* load, one client can submit 20 transactions/sec to
the server.
In this scenario, I noticed that, sometimes, many commands were taking
almost two orders of magnitude more than under the normal load. I used
SQL Profiler to monitor all Statements and SPs taking longer tha 100
msec and, to my surprise, found that, every so often, some command or
SP would take more than 1-2 seconds. What is interesting is that many
of these commands are IF @.@.TRANCOUNT > 0 COMMIT TRAN, which, I think,
ADO implicitly sends to the server. Those usually show with duration 0
(Zero) under normal load.
What I wanted from you, were some ideas on how to go about
troubleshooting this problem, by identifying the underlying cause for
such poor performance. The problem does not seem associated with a
particular command or SP. It also does not seem related to CPU
contention on the server because it is kept really low (about 20%).
Your help is greatly appreciated.
- CDThis is a multi-part message in MIME format.
--=_NextPart_000_0074_01C3EB4D.F4E4CB70
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
How necessary is the Begin Tran/End Tran? If you don't need to make a
series of updates atomic then leave this out.
--
Tom
----
Thomas A. Moreau, BSc, PhD, MCSE, MCDBA
SQL Server MVP
Columnist, SQL Server Professional
Toronto, ON Canada
www.pinnaclepublishing.com/sql
.
<crbd98@.yahoo.com> wrote in message
news:edf41e63.0402041449.724d721b@.posting.google.com...
Greetings,
In our current project, we use a combination of SQL Server 2000 and
ADO. Our database is very simple and small: just about 10 tables with
300K records at most (in just one of the tables). We need, however,
very fast responses to our queries. With that in mind, we designed and
optimized all queries and indices in such a way that each query takes
less that 20 milliseconds, as measured using the SQL profiler under a
normal load.
Each client application opens a single connection to the database and
the queries are funneled through that connection. Each query is a
individual transaction, i.e. it is fenced by Begin Tran...End Tran. We
use mostly stored procedures, which are executed via the _Command
object from ADO. In just a couple of cases, we use _Recordset.
Under a *stress* load, one client can submit 20 transactions/sec to
the server.
In this scenario, I noticed that, sometimes, many commands were taking
almost two orders of magnitude more than under the normal load. I used
SQL Profiler to monitor all Statements and SPs taking longer tha 100
msec and, to my surprise, found that, every so often, some command or
SP would take more than 1-2 seconds. What is interesting is that many
of these commands are IF @.@.TRANCOUNT > 0 COMMIT TRAN, which, I think,
ADO implicitly sends to the server. Those usually show with duration 0
(Zero) under normal load.
What I wanted from you, were some ideas on how to go about
troubleshooting this problem, by identifying the underlying cause for
such poor performance. The problem does not seem associated with a
particular command or SP. It also does not seem related to CPU
contention on the server because it is kept really low (about 20%).
Your help is greatly appreciated.
- CD
--=_NextPart_000_0074_01C3EB4D.F4E4CB70
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
&

How necessary is the Begin Tran/End =Tran? If you don't need to make a series of updates atomic then leave this out.
-- Tom
----Thomas A. =Moreau, BSc, PhD, MCSE, MCDBASQL Server MVPColumnist, SQL Server ProfessionalToronto, ON Canadahttp://www.pinnaclepublishing.com/sql">www.pinnaclepublishing.com=/sql.
=wrote in message news:edf41e=63.0402041449.724d721b@.posting.google.com...Greetings,I=n our current project, we use a combination of SQL Server 2000 andADO. =Our database is very simple and small: just about 10 tables with300K =records at most (in just one of the tables). We need, however,very fast =responses to our queries. With that in mind, we designed andoptimized all queries =and indices in such a way that each query takesless that 20 =milliseconds, as measured using the SQL profiler under anormal load.Each =client application opens a single connection to the database andthe queries =are funneled through that connection. Each query is aindividual =transaction, i.e. it is fenced by Begin Tran...End Tran. Weuse mostly stored =procedures, which are executed via the _Commandobject from ADO. In just a couple =of cases, we use _Recordset.Under a *stress* load, one client can =submit 20 transactions/sec tothe server.In this scenario, I noticed that, sometimes, many commands were takingalmost two orders of magnitude =more than under the normal load. I usedSQL Profiler to monitor all Statements =and SPs taking longer tha 100msec and, to my surprise, found that, every so =often, some command orSP would take more than 1-2 seconds. What is =interesting is that manyof these commands are IF @.@.TRANCOUNT > 0 COMMIT TRAN, =which, I think,ADO implicitly sends to the server. Those usually show with =duration 0(Zero) under normal load.What I wanted from you, were some =ideas on how to go abouttroubleshooting this problem, by identifying the =underlying cause forsuch poor performance. The problem does not seem associated =with aparticular command or SP. It also does not seem related to CPUcontention on the server because it is kept really low (about 20%).Your help is greatly appreciated.- CD

--=_NextPart_000_0074_01C3EB4D.F4E4CB70--|||sounds like some blocking issues.
when you run your tests, monitor blocks in SQL Server (and\or Deadlocks)
do your sprocs have NOLOCK Hints in them when appropriate ?
Or setting transaction isolation level to READ UNCOMMITTED (Where
Appropriate).
probably worth looking into.
cheers
Greg Jackson
PDX, OR|||Are you also using VB transactions? I found performance
improvement by issuing transactions only from my stored
procedures. I believe VB trans cause SQL to SET
IMPLICIT_TRANSACTIONS ON which can increase lock
contention & network round trips. I also saw
improvements by using disconnected recordsets.
Darren Fuller
SQL Server DBA MCSE
>--Original Message--
>Greetings,
>In our current project, we use a combination of SQL
Server 2000 and
>ADO. Our database is very simple and small: just about
10 tables with
>300K records at most (in just one of the tables). We
need, however,
>very fast responses to our queries. With that in mind,
we designed and
>optimized all queries and indices in such a way that
each query takes
>less that 20 milliseconds, as measured using the SQL
profiler under a
>normal load.
>Each client application opens a single connection to the
database and
>the queries are funneled through that connection. Each
query is a
>individual transaction, i.e. it is fenced by Begin
Tran...End Tran. We
>use mostly stored procedures, which are executed via the
_Command
>object from ADO. In just a couple of cases, we use
_Recordset.
>Under a *stress* load, one client can submit 20
transactions/sec to
>the server.
>In this scenario, I noticed that, sometimes, many
commands were taking
>almost two orders of magnitude more than under the
normal load. I used
>SQL Profiler to monitor all Statements and SPs taking
longer tha 100
>msec and, to my surprise, found that, every so often,
some command or
>SP would take more than 1-2 seconds. What is interesting
is that many
>of these commands are IF @.@.TRANCOUNT > 0 COMMIT TRAN,
which, I think,
>ADO implicitly sends to the server. Those usually show
with duration 0
>(Zero) under normal load.
>What I wanted from you, were some ideas on how to go
about
>troubleshooting this problem, by identifying the
underlying cause for
>such poor performance. The problem does not seem
associated with a
>particular command or SP. It also does not seem related
to CPU
>contention on the server because it is kept really low
(about 20%).
>Your help is greatly appreciated.
>- CD
>.
>

Degrading performance. Expert help appreciated

Greetings,
In our current project, we use a combination of SQL Server 2000 and
ADO. Our database is very simple and small: just about 10 tables with
300K records at most (in just one of the tables). We need, however,
very fast responses to our queries. With that in mind, we designed and
optimized all queries and indices in such a way that each query takes
less that 20 milliseconds, as measured using the SQL profiler under a
normal load.
Each client application opens a single connection to the database and
the queries are funneled through that connection. Each query is a
individual transaction, i.e. it is fenced by Begin Tran...End Tran. We
use mostly stored procedures, which are executed via the _Command
object from ADO. In just a couple of cases, we use _Recordset.
Under a *stress* load, one client can submit 20 transactions/sec to
the server.
In this scenario, I noticed that, sometimes, many commands were taking
almost two orders of magnitude more than under the normal load. I used
SQL Profiler to monitor all Statements and SPs taking longer tha 100
msec and, to my surprise, found that, every so often, some command or
SP would take more than 1-2 seconds. What is interesting is that many
of these commands are IF @.@.TRANCOUNT > 0 COMMIT TRAN, which, I think,
ADO implicitly sends to the server. Those usually show with duration 0
(Zero) under normal load.
What I wanted from you, were some ideas on how to go about
troubleshooting this problem, by identifying the underlying cause for
such poor performance. The problem does not seem associated with a
particular command or SP. It also does not seem related to CPU
contention on the server because it is kept really low (about 20%).
Your help is greatly appreciated.
- CDHow necessary is the Begin Tran/End Tran? If you don't need to make a
series of updates atomic then leave this out.
Tom
----
Thomas A. Moreau, BSc, PhD, MCSE, MCDBA
SQL Server MVP
Columnist, SQL Server Professional
Toronto, ON Canada
www.pinnaclepublishing.com/sql
.
<crbd98@.yahoo.com> wrote in message
news:edf41e63.0402041449.724d721b@.posting.google.com...
Greetings,
In our current project, we use a combination of SQL Server 2000 and
ADO. Our database is very simple and small: just about 10 tables with
300K records at most (in just one of the tables). We need, however,
very fast responses to our queries. With that in mind, we designed and
optimized all queries and indices in such a way that each query takes
less that 20 milliseconds, as measured using the SQL profiler under a
normal load.
Each client application opens a single connection to the database and
the queries are funneled through that connection. Each query is a
individual transaction, i.e. it is fenced by Begin Tran...End Tran. We
use mostly stored procedures, which are executed via the _Command
object from ADO. In just a couple of cases, we use _Recordset.
Under a *stress* load, one client can submit 20 transactions/sec to
the server.
In this scenario, I noticed that, sometimes, many commands were taking
almost two orders of magnitude more than under the normal load. I used
SQL Profiler to monitor all Statements and SPs taking longer tha 100
msec and, to my surprise, found that, every so often, some command or
SP would take more than 1-2 seconds. What is interesting is that many
of these commands are IF @.@.TRANCOUNT > 0 COMMIT TRAN, which, I think,
ADO implicitly sends to the server. Those usually show with duration 0
(Zero) under normal load.
What I wanted from you, were some ideas on how to go about
troubleshooting this problem, by identifying the underlying cause for
such poor performance. The problem does not seem associated with a
particular command or SP. It also does not seem related to CPU
contention on the server because it is kept really low (about 20%).
Your help is greatly appreciated.
- CD|||sounds like some blocking issues.
when you run your tests, monitor blocks in SQL Server (and\or Deadlocks)
do your sprocs have NOLOCK Hints in them when appropriate ?
Or setting transaction isolation level to READ UNCOMMITTED (Where
Appropriate).
probably worth looking into.
cheers
Greg Jackson
PDX, ORsql

Sunday, March 11, 2012

Defrag and Index

I am by no means an SQL expert, I need to figure out if doing a nightly
reindex defrags my indexes. From the data below it seems pretty obvious it
doesn't, but I am unsure.
I ran a DBCC SHOWCONTIG and I have a index that appears to be absolutely a
mess. Is this correct?
Will running DBCC INDEXDEFRAG help? Accordingly, the documentation is less
than enthusiastic about doing an online defrag.
Do I need to do an DBCC DBREINDEX
So to recap...
1) Does night reindex, defrag?
2) Is the index below a disaster?
3) What is the best way to get this defragged? Keeping the index online
is preferred, but not necessarily required.
========================================================
DBCC SHOWCONTIG scanning 'TS1Endpoints' table...
Table: 'TS1Endpoints' (1860201677); index ID: 0, database ID: 10
TABLE level scan performed.
- Pages Scanned........................: 948
- Extents Scanned.......................: 204
- Extent Switches.......................: 203
- Avg. Pages per Extent..................: 4.6
- Scan Density [Best Count:Actual Count]......: 58.33% [119:204]
- Extent Scan Fragmentation ...............: 99.51%
- Avg. Bytes Free per Page................: 889.5
- Avg. Page Density (full)................: 89.01%
DBCC SHOWCONTIG scanning 'TS1Endpoints' table...
Table: 'TS1Endpoints' (1860201677); index ID: 2, database ID: 10
LEAF level scan performed.
- Pages Scanned........................: 292
- Extents Scanned.......................: 37
- Extent Switches.......................: 36
- Avg. Pages per Extent..................: 7.9
- Scan Density [Best Count:Actual Count]......: 100.00% [37:37]
- Logical Scan Fragmentation ..............: 0.00%
- Extent Scan Fragmentation ...............: 0.00%
- Avg. Bytes Free per Page................: 810.7
- Avg. Page Density (full)................: 89.98%
--
Paul Bergson
MVP - Directory Services
MCT, MCSE, MCSA, Security+, BS CSci
2003, 2000 (Early Achiever), NT
http://www.pbbergs.com
Please no e-mails, any questions should be posted in the NewsGroup
This posting is provided "AS IS" with no warranties, and confers no rights.Paul,
There are not to many pages, so I will not worry much. If you really want to
defrag that table, then you have to add a clustered index, or dump the data,
recreate the table and import the data.
AMB
"Paul Bergson [MVP-DS]" wrote:
> I am by no means an SQL expert, I need to figure out if doing a nightly
> reindex defrags my indexes. From the data below it seems pretty obvious it
> doesn't, but I am unsure.
> I ran a DBCC SHOWCONTIG and I have a index that appears to be absolutely a
> mess. Is this correct?
> Will running DBCC INDEXDEFRAG help? Accordingly, the documentation is less
> than enthusiastic about doing an online defrag.
> Do I need to do an DBCC DBREINDEX
> So to recap...
> 1) Does night reindex, defrag?
> 2) Is the index below a disaster?
> 3) What is the best way to get this defragged? Keeping the index online
> is preferred, but not necessarily required.
> ========================================================> DBCC SHOWCONTIG scanning 'TS1Endpoints' table...
> Table: 'TS1Endpoints' (1860201677); index ID: 0, database ID: 10
> TABLE level scan performed.
> - Pages Scanned........................: 948
> - Extents Scanned.......................: 204
> - Extent Switches.......................: 203
> - Avg. Pages per Extent..................: 4.6
> - Scan Density [Best Count:Actual Count]......: 58.33% [119:204]
> - Extent Scan Fragmentation ...............: 99.51%
> - Avg. Bytes Free per Page................: 889.5
> - Avg. Page Density (full)................: 89.01%
> DBCC SHOWCONTIG scanning 'TS1Endpoints' table...
> Table: 'TS1Endpoints' (1860201677); index ID: 2, database ID: 10
> LEAF level scan performed.
> - Pages Scanned........................: 292
> - Extents Scanned.......................: 37
> - Extent Switches.......................: 36
> - Avg. Pages per Extent..................: 7.9
> - Scan Density [Best Count:Actual Count]......: 100.00% [37:37]
> - Logical Scan Fragmentation ..............: 0.00%
> - Extent Scan Fragmentation ...............: 0.00%
> - Avg. Bytes Free per Page................: 810.7
> - Avg. Page Density (full)................: 89.98%
>
> --
> Paul Bergson
> MVP - Directory Services
> MCT, MCSE, MCSA, Security+, BS CSci
> 2003, 2000 (Early Achiever), NT
> http://www.pbbergs.com
> Please no e-mails, any questions should be posted in the NewsGroup
> This posting is provided "AS IS" with no warranties, and confers no rights.
>
>|||So the line
Extent Scan Fragmentation ...............: 99.51%
Is not bad?
--
Paul Bergson
MVP - Directory Services
MCT, MCSE, MCSA, Security+, BS CSci
2003, 2000 (Early Achiever), NT
http://www.pbbergs.com
Please no e-mails, any questions should be posted in the NewsGroup
This posting is provided "AS IS" with no warranties, and confers no rights.
"Alejandro Mesa" <AlejandroMesa@.discussions.microsoft.com> wrote in message
news:C52B9490-28EE-4CDD-AF1B-3C8B135FD898@.microsoft.com...
> Paul,
> There are not to many pages, so I will not worry much. If you really want
> to
> defrag that table, then you have to add a clustered index, or dump the
> data,
> recreate the table and import the data.
>
> AMB
> "Paul Bergson [MVP-DS]" wrote:
>> I am by no means an SQL expert, I need to figure out if doing a nightly
>> reindex defrags my indexes. From the data below it seems pretty obvious
>> it
>> doesn't, but I am unsure.
>> I ran a DBCC SHOWCONTIG and I have a index that appears to be absolutely
>> a
>> mess. Is this correct?
>> Will running DBCC INDEXDEFRAG help? Accordingly, the documentation is
>> less
>> than enthusiastic about doing an online defrag.
>> Do I need to do an DBCC DBREINDEX
>> So to recap...
>> 1) Does night reindex, defrag?
>> 2) Is the index below a disaster?
>> 3) What is the best way to get this defragged? Keeping the index
>> online
>> is preferred, but not necessarily required.
>> ========================================================>> DBCC SHOWCONTIG scanning 'TS1Endpoints' table...
>> Table: 'TS1Endpoints' (1860201677); index ID: 0, database ID: 10
>> TABLE level scan performed.
>> - Pages Scanned........................: 948
>> - Extents Scanned.......................: 204
>> - Extent Switches.......................: 203
>> - Avg. Pages per Extent..................: 4.6
>> - Scan Density [Best Count:Actual Count]......: 58.33% [119:204]
>> - Extent Scan Fragmentation ...............: 99.51%
>> - Avg. Bytes Free per Page................: 889.5
>> - Avg. Page Density (full)................: 89.01%
>> DBCC SHOWCONTIG scanning 'TS1Endpoints' table...
>> Table: 'TS1Endpoints' (1860201677); index ID: 2, database ID: 10
>> LEAF level scan performed.
>> - Pages Scanned........................: 292
>> - Extents Scanned.......................: 37
>> - Extent Switches.......................: 36
>> - Avg. Pages per Extent..................: 7.9
>> - Scan Density [Best Count:Actual Count]......: 100.00% [37:37]
>> - Logical Scan Fragmentation ..............: 0.00%
>> - Extent Scan Fragmentation ...............: 0.00%
>> - Avg. Bytes Free per Page................: 810.7
>> - Avg. Page Density (full)................: 89.98%
>>
>> --
>> Paul Bergson
>> MVP - Directory Services
>> MCT, MCSE, MCSA, Security+, BS CSci
>> 2003, 2000 (Early Achiever), NT
>> http://www.pbbergs.com
>> Please no e-mails, any questions should be posted in the NewsGroup
>> This posting is provided "AS IS" with no warranties, and confers no
>> rights.
>>|||Paul,
That number is not relevant to heaps (tables without clustered index) and is
meaningless when the index spans multiple files.
DBCC SHOWCONTIG
http://msdn2.microsoft.com/en-US/library/aa258803(SQL.80).aspx
Microsoft SQL Server 2000 Index Defragmentation Best Practices
http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/ss2kidbp.mspx
AMB
"Paul Bergson [MVP-DS]" wrote:
> So the line
> Extent Scan Fragmentation ...............: 99.51%
> Is not bad?
> --
> Paul Bergson
> MVP - Directory Services
> MCT, MCSE, MCSA, Security+, BS CSci
> 2003, 2000 (Early Achiever), NT
> http://www.pbbergs.com
> Please no e-mails, any questions should be posted in the NewsGroup
> This posting is provided "AS IS" with no warranties, and confers no rights.
> "Alejandro Mesa" <AlejandroMesa@.discussions.microsoft.com> wrote in message
> news:C52B9490-28EE-4CDD-AF1B-3C8B135FD898@.microsoft.com...
> > Paul,
> >
> > There are not to many pages, so I will not worry much. If you really want
> > to
> > defrag that table, then you have to add a clustered index, or dump the
> > data,
> > recreate the table and import the data.
> >
> >
> > AMB
> >
> > "Paul Bergson [MVP-DS]" wrote:
> >
> >> I am by no means an SQL expert, I need to figure out if doing a nightly
> >> reindex defrags my indexes. From the data below it seems pretty obvious
> >> it
> >> doesn't, but I am unsure.
> >>
> >> I ran a DBCC SHOWCONTIG and I have a index that appears to be absolutely
> >> a
> >> mess. Is this correct?
> >>
> >> Will running DBCC INDEXDEFRAG help? Accordingly, the documentation is
> >> less
> >> than enthusiastic about doing an online defrag.
> >>
> >> Do I need to do an DBCC DBREINDEX
> >>
> >> So to recap...
> >> 1) Does night reindex, defrag?
> >> 2) Is the index below a disaster?
> >> 3) What is the best way to get this defragged? Keeping the index
> >> online
> >> is preferred, but not necessarily required.
> >>
> >> ========================================================> >>
> >> DBCC SHOWCONTIG scanning 'TS1Endpoints' table...
> >>
> >> Table: 'TS1Endpoints' (1860201677); index ID: 0, database ID: 10
> >>
> >> TABLE level scan performed.
> >>
> >> - Pages Scanned........................: 948
> >>
> >> - Extents Scanned.......................: 204
> >>
> >> - Extent Switches.......................: 203
> >>
> >> - Avg. Pages per Extent..................: 4.6
> >>
> >> - Scan Density [Best Count:Actual Count]......: 58.33% [119:204]
> >>
> >> - Extent Scan Fragmentation ...............: 99.51%
> >>
> >> - Avg. Bytes Free per Page................: 889.5
> >>
> >> - Avg. Page Density (full)................: 89.01%
> >>
> >> DBCC SHOWCONTIG scanning 'TS1Endpoints' table...
> >>
> >> Table: 'TS1Endpoints' (1860201677); index ID: 2, database ID: 10
> >>
> >> LEAF level scan performed.
> >>
> >> - Pages Scanned........................: 292
> >>
> >> - Extents Scanned.......................: 37
> >>
> >> - Extent Switches.......................: 36
> >>
> >> - Avg. Pages per Extent..................: 7.9
> >>
> >> - Scan Density [Best Count:Actual Count]......: 100.00% [37:37]
> >>
> >> - Logical Scan Fragmentation ..............: 0.00%
> >>
> >> - Extent Scan Fragmentation ...............: 0.00%
> >>
> >> - Avg. Bytes Free per Page................: 810.7
> >>
> >> - Avg. Page Density (full)................: 89.98%
> >>
> >>
> >> --
> >> Paul Bergson
> >> MVP - Directory Services
> >> MCT, MCSE, MCSA, Security+, BS CSci
> >> 2003, 2000 (Early Achiever), NT
> >>
> >> http://www.pbbergs.com
> >>
> >> Please no e-mails, any questions should be posted in the NewsGroup
> >> This posting is provided "AS IS" with no warranties, and confers no
> >> rights.
> >>
> >>
> >>
>
>|||Paul Bergson [MVP-DS] (pbergson@.allete_nospam.com) writes:
> I am by no means an SQL expert, I need to figure out if doing a nightly
> reindex defrags my indexes. From the data below it seems pretty obvious
> it doesn't, but I am unsure.
> I ran a DBCC SHOWCONTIG and I have a index that appears to be absolutely a
> mess. Is this correct?
No, because that's not an index, but a heap, a table without a
clustered index. Heaps are quite prone to fragmentation, and your
table is not in the best shape with 4.6 pages per extent. But as
Alejandro pointed out the table is not that big, and it is not likely
to be a major problem.
> Will running DBCC INDEXDEFRAG help? Accordingly, the documentation is
> less than enthusiastic about doing an online defrag.
Neither INDEXDEFRAG or DBREINDEX works on heap. You can build a clustered
index on the table and then drop it. Or you can just build a clustered index
and have it that way.
Erland Sommarskog, SQL Server MVP, esquel@.sommarskog.se
Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/prodtechnol/sql/2005/downloads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinfo/previousversions/books.mspx