Skip to content
accelerando

Tag Archives: SQL

Optimizing the Oracle Query Optimizer

20-Oct-10

The Oracle Query Optimizer does a great job in rewriting Queries. For example the nested subquery in

SELECT C.cust_last_name, C.country_id 
FROM    customers C 
WHERE EXISTS (SELECT 1 
                          FROM sales S 
                          WHERE S.quantity_sold > 1000 AND 
                                        S.cust_id = C.cust_id);

will be rewritten into the following query, using a semi-join:

SELECT C.cust_last_name, C.country_id 
FROM customers C, sales S 
WHERE S.quantity_sold > 1000 AND 
C.cust_id S= S.cust_id;

(Example from Optimizer Transformations: Subquery Unnesting part 1)

In the first form, the subquery will be invoked for each entry in customers, replacing the value of the correlated column C.cust_id. Thus the query cannot be parallelized among other implications. Most of the time, if the columns are correctly indexed, the semijoin will be much faster.

This form of rewriting is called “unnesting”.

Consider the following query:

SELECT C.cust_last_name, C.country_id 
FROM    customers C, (SELECT f_compute_some_complicated_value(s.quantity_sold), S.cust_id
                          FROM sales S                          
                          WHERE S.quantity_sold > 1000) S
WHERE  S.cust_id = C.cust_id

The subquery in the from clause its called a derived table. It will be rewritten as:

SELECT C.cust_last_name, C.country_id, f_compute_some_complicated_value(s.quantity_sold), S.cust_id
FROM    customers C, Sales S
WHERE  S.cust_id = C.cust_id 
AND S.quantity_sold > 1000)

This technique is called query merging. Here the join should also be faster in most cases.

Both techniques are good and will improve many queries.

But consider a scenario where the subqueries query remote tables (via database link for example) or call PL/SQL or even Java stored procedures and the main table (customes in the examples) contains many more rows than the joined table or the where clause is less selective then the where clause of the subquery: The subquery will be executed as many times as there are rows in the main table. This can lead to very poor performance.

Luckily, the optimizer can be given hints:

To stop unnesting, use the NO_UNNEST hint:

SELECT C.cust_last_name, C.country_id 
FROM    customers C 
WHERE EXISTS (SELECT /*+ NO_UNNEST */ 1 
                          FROM sales S 
                          WHERE S.quantity_sold > 1000 AND 
                                        S.cust_id = C.cust_id);

Furthermore, i’d rewrite the query like so:

SELECT C.cust_last_name, C.country_id 
FROM    customers C 
WHERE C.cust_id =ANY (SELECT /*+ NO_UNNEST */ S.cust_id 
                                       FROM sales S 
                                       WHERE S.quantity_sold > 1000);

Where the EXISTS clause triggers the subquery for each item in customes, the =ANY clause will trigger the subquery only once. Both queries are semantically identical.

To stop the merging of derived tables, use the NO_MERGE hint in the surrounding select clause:

SELECT /*+ NO_MERGE(S) */ C.cust_last_name, C.country_id 
FROM    customers C, (SELECT f_compute_some_complicated_value(s.quantity_sold), S.cust_id
                          FROM sales S                          
                          WHERE S.quantity_sold > 1000) S
WHERE  S.cust_id = C.cust_id

The NO_MERGE hint takes the alias of the derived table as a parameter.

Combining both techniques brought a complex query down from 40s to just about 1s. We are using an Oracle 10g Database, but the hints also apply to 11g.

Oracle NLS-Lang Settings

13-Aug-10

To use SQL*Plus Windows correctly, export NLS_LANG like so:

set NLS_LANG=GERMAN_GERMANY.WE8MSWIN1252

If you prefer the command line client SQL*Plus, export NLS_LANG like so:

set NLS_LANG=GERMAN_GERMANY.WE8PC850

Using SQL*Plus without tnsnames.ora

29-Mar-10

As often, a quick reminder for me. I’m using the Oracle Instaclient on my Mac without a tnsnames.ora and i keep forgetting the connectstring syntax:

sqlplus USER/PASS@//HOST:PORT/SID

Extra bonus points: Through in rlwrap to get a nice commandline history and completion as used to in a standard shell:

rlwrap sqlplus USER/PASS@//HOST:PORT/SID

Fun with sql

27-Oct-08

What’s all the fuss about this SQL Injection thing?

It boils down getting some malicious crafted SQL code into the SQL code of an application, destroying data or authenticate yourself without knowing any real password. xkdc has a nice explanation.

The simple cases base on wrong escaped strings and the like. But as this SQL injection cheatsheet shows there are an infinity number of possibilities.

At day most of the time my database connection is an Oracle connection and so i found this Oracle whitepaper titled How to write injection-proof PL/SQL very interesting (via Bruce Schneier found at the gay bar).

I do not have a magic recipe for avoiding attack vectors all the time but as well as the whitepaper is written, it’s not a solution to expose all queries only via pl/sql to clients. In fact, it’s a nightmare to get this to work with JPA and other ORM mappers.

I try not to use dynamic sql in the sense of “concatenate some strings with one another and mysql_real_escape_string or DBMS_Assert. them” but use prepared statements with placeholders and explicit datatypes. Also if there’s a need for computing sql queries at runtime, do not ever let user supplied input come near them. I know that i’m relying to my api in this case but there is always a point on which i must rely on i guess.

As alway, the most important thing is: Be conscious about what you are doing and try to understand that, but at this point, i leave the discussion about software development and enter the depths of common sense…

Close
E-mail It