Database Programming: Set-Based Update — Hugo’s Syntax Outperforms Shaun’s; Ward Tweaks Hugo’s Syntax Even Further

This is turning into a good old-fashioned coding party.  I love it..

Hugo Kornelis left a wonderful comment on last night's post (thanks Hugo!) answering my challenge to outperform Shaun's syntax.  With some adjustments I've made to control for parallelism (we can argue later whether that's a good thing; when I leave parallelism in play on my sandbox (which turns out to be hyperthreaded, contrary to what I said last night) the results of the following tests are far too random), I have some results to share..

I expanded on Hugo's approach and built 5 possible approaches to this problem (if you've got a different one, bring it on..).  Let's name them as follows (along with the average of five execution times on my laptop (HP nc8430; 2gB RAM):

• 1MRowTable (2156 ms)
• ShaunsCTE (1546 ms)
• HugosFourWayJoin (1123 ms)
• HugosTwoWayJoin (1110 ms)
• TwoWayJoinNoTOP (983 ms)

Here's a rundown on each alternative..

1MRowTable

Looking at Hugo's syntax, I wondered if we might get more efficiency from a SELECT off of a 1M row table than off of a four-way Cartesian product.  This requires a dedicated table, which we'll populate with an adaptation of Shaun's CTE:

drop table SetBuilder

go

create table SetBuilder (Id bigint PRIMARY KEY)

go

with digits(i) as (

    select 1 as i union all select 2 union all select 3 union all
    select 4 union all select 5 union all select 6 union all
    select 7 union all select 8 union all select 9 union all
    select 0
),
-- generate 1M rows each with a unique row number, i
sequence(i) as (
  select d1.i + (10*d2.i) + (100*d3.i) + (1000*d4.i) + (10000*d5.i) + (100000*d6.i)
      from digits as d1,
           digits as d2,
           digits as d3,
           digits as d4,
           digits as d5,
           digits as d6
)

insert SetBuilder (Id)

select i
from sequence 

Here's the test for generating 1M GUIDs using the table we just populated:

drop table SomeTable;

go
checkpoint;
dbcc freeproccache;
dbcc dropcleanbuffers;
go

declare @start datetime,

          @datediff int;
set @start = getdate();

select newid() as uuid into SomeTable
from SetBuilder
option (maxdop 1);

select @datediff = datediff(ms, @start, getdate());

select @datediff  AS [1MRowTable], count(*)

from SomeTable;

ShaunsCTE

Here's Shaun's CTE blended with Hugo's monitoring syntax:

drop table SomeTable;

go
checkpoint;
dbcc freeproccache;
dbcc dropcleanbuffers;
go

declare @start datetime,

          @datediff int;

set @start = getdate();

with digits(i) as (

    select 1 as i union all select 2 union all select 3 union all
    select 4 union all select 5 union all select 6 union all
    select 7 union all select 8 union all select 9 union all
    select 0
),

-- generate 1M rows each with a unique row number, i

sequence(i) as (

 select d1.i + (10*d2.i) + (100*d3.i) + (1000*d4.i) + (10000*d5.i) + (100000*d6.i)
 from digits as d1, digits as d2, digits as d3, digits as d4, digits as d5, digits as d6
)

select newid() as uuid into SomeTable

from sequence option (maxdop 1);

select @datediff = datediff(ms, @start, getdate());

select @datediff  AS ShaunsCTE, count(*)

from SomeTable;

HugosFourWayJoin

Here's Hugo's four-way JOIN proposal, appropriate for use in a newly created database:

drop table SomeTable;

go
checkpoint;
dbcc freeproccache;
dbcc dropcleanbuffers;
go

declare @start datetime,

          @datediff int;

set @start = getdate();

select top (1000000) newid() as uuid

into SomeTable
from sysobjects a, sysobjects b, sysobjects c, sysobjects d;
select @datediff = datediff(ms, @start, getdate());
select @datediff  AS HugosFourWayJoin, count(*)
from SomeTable option (maxdop 1);

HugosTwoWayJoin

Here's Hugo's two-way JOIN, appropriate for use in databases with several user objects defined:

drop table SomeTable;

go

checkpoint;
dbcc freeproccache;
dbcc dropcleanbuffers;
go

declare @start datetime,

          @datediff int;

set @start = getdate();

select top (1000000) newid() as uuid into SomeTable

from sysobjects a, sysobjects b

select @datediff = datediff(ms, @start, getdate());

select @datediff  AS HugosTwoWayJoin, count(*)

from SomeTable option (maxdop 1);

WardsTwoWayJoinNoTOP

Given that the two-way JOIN outperformed the four-way JOIN, I thought we might get better performance if we built a contrived table containing 1000 rows, which would generate an exactly 1M row Cartesian product, which removes the need for the TOP clause in the SELECT:

create table SetBuilder1K (Id bigint PRIMARY KEY)

go

;with digits(i) as (

    select 1 as i union all select 2 union all select 3 union all
    select 4 union all select 5 union all select 6 union all
    select 7 union all select 8 union all select 9 union all
    select 0
),
-- generate 1M rows each with a unique row number, i
sequence(i) as (
  select d1.i + (10*d2.i) + (100*d3.i) + (1000*d4.i) + (10000*d5.i) + (100000*d6.i)
      from digits as d1,
           digits as d2,
           digits as d3,
           digits as d4,
           digits as d5,
           digits as d6
)

insert SetBuilder1K (Id)

select TOP 1000 i
from sequence
order by i

drop table SomeTable;

go

checkpoint;

dbcc freeproccache;
dbcc dropcleanbuffers;

go

declare @start datetime,
          @datediff int;

set @start = getdate();

select newid() as uuid into SomeTable

from SetBuilder1K a, SetBuilder1K b

select @datediff = datediff(ms, @start, getdate());

select @datediff  AS WardsTwoWayJoinNoTOP, count(*)

from SomeTable option (maxdop 1);

Let's Keep This Party Going!

It looks as though the precisely contrived Cartesian product is the most performant of these alternatives.  This is further demonstration, I think, that it's best to let SQL Server be SQL Server; working with tables appears to be more performant than working with CTEs.

This is definitely a work in progress, and I hope those of you who are so inclined will join the party and throw some code on the bonfire.  I'd be especially interested in knowing if anybody can get consistent results with parallelism enabled.

If you've made it this far, thanks..

     -wp

PS.  If you have made it this far, your reward is the name of Hugo's blog, which I think is so brilliant, I'm going to pass it through the SQL query parser:

SELECT  Hints,

        Tips,
        Tricks
FROM    Hugo Kornelis
WHERE   RDBMS = 'SQL Server'

Reference: https://blogs.technet.microsoft.com/wardpond/2006/09/21/database-programming-set-based-update-hugos-syntax-outperforms-shauns-ward-tweaks-hugos-syntax-even-further/