PostgreSQL on Stefan Litsche

Sorting in Lakes

Fri, 02 Jul 2021 10:26:00 +0100

In case we sort rows using SQL we need to distinguish between duplicate values of unique rows and duplicate values due to duplicate rows.

The nice thing about the row_number SQL function is that it provides you with a unique number (per partition) which is often used as a mean to de-duplicate a set. The rank function does not provide this property because duplicate values share the same rank.

SELECT a,
    dense_rank() OVER (ORDER BY a),
    rank() OVER (ORDER BY a),
    row_number() OVER (ORDER BY a)
FROM (VALUES (1),(1),(2)) AS t(a);

 a | dense_rank | rank | row_number
---+------------+------+------------
 1 |          1 |    1 |          1
 1 |          1 |    1 |          2
 2 |          2 |    3 |          3

The problem occurs if you use it the wrong way. Recently I spent a some time debugging a weird behavior in a report. The numbers changed during different executions on an immutable data set — which is not what you expect.

An Update on Statistics Target

Sun, 01 Dec 2019 10:30:00 +0100

In an earlier article I analyzed the influence of the statistics target on the result of sampling for extreme distributions. The representation of extreme rare values in the most common values required a drastic increase of the sample size.

My colleage Alex Shulgin initiated a patch which improved the situation for null values. In PostgreSQL 9.6 the improvements for analyze were released. More work was done on this issue later to improve the selection of most common values which was released in PostgreSQL 11.

Analyze Extreme Distributions in Postgresql

Thu, 30 Jul 2015 10:30:00 +0100

Recently my team and I observed in our PostgreSQL databases a sporadic increase in the execution time of stored procedures (see the graph above). Often it happened that an analyze of the referenced table solved the issue. In our case, fluctuations in our execution plan caused statement timeouts. This led to errors in our applications.

We wanted to understand this behavior better. Which circumstances prompted more frequent plan fluctuations? How exactly could we influence the system to be more reliable? To find answers, we tested how different configurations of PostgreSQL influenced the results of the query planner. This post shares the results of our tests.

PostgreSQL Background Writer Statistics

Tue, 07 Oct 2014 10:30:00 +0100

The pg_stat_bgwriter view provides some interesting information about the background writer. This view provides only one row. If you want to analyze these data, these results has to be stored somewhere with a timestamp. The following shall show that it allows an insight into the changes performed in shared buffer cache and also hints for tuning several server parameters.

If a backend process (a client) requests data it is either found in a block in shared buffer cache or the block has to be allocated (read from disk). The latter is counted in buffers_alloc column. The usage count of this block is increased. The backend process modifies data in this requested page, so this page becomes dirty. With the commit the transaction is written to WAL file. At some moment the background writer will synchronize the dirty page with the disk and mark this page as clean.

Umbennen von Postgres Typen

Thu, 08 Nov 2012 16:49:57 +0000

In PL/pgSQL, eine der Programmiersprachen für die serverseitige Programmierung in PostgreSQL, werden das Interface der Prozedur und Body unterschiedlich behandelt. Der Body wird beim ersten Aufruf in einer Session kompiliert, für SQL werden Prepared Statements und ggfs. auch Ausführungspläne erstellt und diese im Cache bis zum Ende der Session gespeichert. Innerhalb eines Bedingungsbaums werden so nur die Statements vorbereitet, die auch benötigt werden (siehe Dokumentation).

Typen können in PostgreSQL erstellt und als Parameter oder Variablen referenziert werden. Typen in Parameterdeklarationen werden beim Erstellen der Funktion in die OID aufgelöst, Typen in Variablen Deklarationen zu Beginn der Session, wenn der Body compiliert wird.

NULL Werte in Order By Klausel

Thu, 21 Jul 2011 16:21:00 +0000

Man möchte in einer nach einer Spalte sortieren die Null Werte enthält. Aber man braucht ein sicheres Verhalten dafür, ob die NULL Werte am Beginn oder am Ende eingefügt werden. Wie kann man das standard konform und backend unabhängig realisieren?

Am einfachsten wäre es, wenn es ein zugesichertes Verhalten gibt. Leider gibt es das nicht. Der Standard gibt nur vor, dass alle Nullmarken gleich behandelt werden. Der ISO Standard SQL/92 sagt dazu (Zitat C.Date, H.Darwin (1998) SQL - Der Standard. Seite 278):