georgiybykov/explicit_locking.md

Last active March 29, 2022 22:37

Star (0) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/georgiybykov/87b57339e6e4d42181286f805867c576.js"></script>
Save georgiybykov/87b57339e6e4d42181286f805867c576 to your computer and use it in GitHub Desktop.

Download ZIP

SQL queries with transactions

Raw

explicit_locking.md

1st console:

udatapp_development=# begin;
BEGIN

udatapp_development=# lock table users in access exclusive mode;
LOCK TABLE

udatapp_development=#

2nd console:

udatapp_development=# begin; UPDATE users SET email = '[email protected]' WHERE id = 2;
BEGIN

3rd console:

udatapp_development=# SELECT locktype, relation::regclass, mode, transactionid AS tid,
virtualtransaction AS vtid, pid, granted
FROM pg_catalog.pg_locks l LEFT JOIN pg_catalog.pg_database db
ON db.oid = l.database WHERE (db.datname = 'udatapp_development' OR db.datname IS NULL)
AND NOT pid = pg_backend_pid();

   locktype    | relation |        mode         |  tid   |  vtid   |   pid   | granted 
---------------+----------+---------------------+--------+---------+---------+---------
 virtualxid    |          | ExclusiveLock       |        | 6/1333  | 3350240 | t
 virtualxid    |          | ExclusiveLock       |        | 5/20859 | 3350531 | t
 relation      | users    | RowExclusiveLock    |        | 6/1333  | 3350240 | f
 relation      | users    | AccessExclusiveLock |        | 5/20859 | 3350531 | t
 transactionid |          | ExclusiveLock       | 818650 | 5/20859 | 3350531 | t
(5 rows)

But there is no transactionid, so it is empty:

udatapp_development=# SELECT blockeda.pid AS blocked_pid, blockeda.query as blocked_query,
  blockinga.pid AS blocking_pid, blockinga.query as blocking_query
FROM pg_catalog.pg_locks blockedl
JOIN pg_stat_activity blockeda ON blockedl.pid = blockeda.pid
JOIN pg_catalog.pg_locks blockingl ON(blockingl.transactionid=blockedl.transactionid
  AND blockedl.pid != blockingl.pid)
JOIN pg_stat_activity blockinga ON blockingl.pid = blockinga.pid
WHERE NOT blockedl.granted AND blockinga.datname='udatapp_development';

 blocked_pid | blocked_query | blocking_pid | blocking_query 
-------------+---------------+--------------+----------------
(0 rows)

We can check the query duration:

udatapp_development=# SELECT pid, query, now() - query_start  AS waiting_duration
FROM pg_catalog.pg_stat_activity WHERE datname='udatapp_development';

   pid   |                                 query                                 |    waiting_duration    
---------+-----------------------------------------------------------------------+------------------------
 3350531 | lock table users in access exclusive mode;                            | 00:02:26.592682
 3350240 | UPDATE users SET email = '[email protected]' WHERE id = 2;    | 00:01:46.382441
 3351199 | SELECT pid, query, now() - query_start  AS waiting_duration          +| 00:00:00
         | FROM pg_catalog.pg_stat_activity WHERE datname='udatapp_development'; | 
(5 rows)

Let's join the relation on which the lock is hold:

udatapp_development=# SELECT blockingl.relation::regclass,
  blockeda.pid AS blocked_pid, blockeda.query as blocked_query,
  blockedl.mode as blocked_mode,
  blockinga.pid AS blocking_pid, blockinga.query as blocking_query,
  blockingl.mode as blocking_mode
FROM pg_catalog.pg_locks blockedl
JOIN pg_stat_activity blockeda ON blockedl.pid = blockeda.pid
JOIN pg_catalog.pg_locks blockingl ON(blockingl.relation=blockedl.relation
  AND blockingl.locktype=blockedl.locktype AND blockedl.pid != blockingl.pid)
JOIN pg_stat_activity blockinga ON blockingl.pid = blockinga.pid
WHERE NOT blockedl.granted AND blockinga.datname='udatapp_development';

 relation | blocked_pid |                           blocked_query                            |   blocked_mode   | blocking_pid |               blocking_query               |    blocking_mode    
----------+-------------+--------------------------------------------------------------------+------------------+--------------+--------------------------------------------+---------------------
 users    |     3350240 | UPDATE users SET email = '[email protected]' WHERE id = 2; | RowExclusiveLock |      3350531 | lock table users in access exclusive mode; | AccessExclusiveLock
(1 row)

1st console:

Commit transaction:

udatapp_development=# commit;
COMMIT

2nd console:

We see that update is completed and we are still inside the transaction:

udatapp_development=# begin; UPDATE users SET email = '[email protected]' WHERE id = 2;
BEGIN
UPDATE 1

So, the real transactionid has been added:

3rd console:

udatapp_development=# SELECT locktype, relation::regclass, mode, transactionid AS tid,
virtualtransaction AS vtid, pid, granted
FROM pg_catalog.pg_locks l LEFT JOIN pg_catalog.pg_database db
ON db.oid = l.database WHERE (db.datname = 'udatapp_development' OR db.datname IS NULL)
AND NOT pid = pg_backend_pid();

   locktype    |       relation       |       mode       |  tid   |  vtid  |   pid   | granted 
---------------+----------------------+------------------+--------+--------+---------+---------
 relation      | index_users_on_email | RowExclusiveLock |        | 6/1333 | 3350240 | t
 relation      | users_pkey           | RowExclusiveLock |        | 6/1333 | 3350240 | t
 virtualxid    |                      | ExclusiveLock    |        | 6/1333 | 3350240 | t
 relation      | users                | RowExclusiveLock |        | 6/1333 | 3350240 | t
 transactionid |                      | ExclusiveLock    | 818651 | 6/1333 | 3350240 | t
(5 rows)

Let's commit transaction in the 2nd console:

2nd console:

udatapp_development=# commit;
COMMIT

3rd console:

udatapp_development=# SELECT locktype, relation::regclass, mode, transactionid AS tid,
virtualtransaction AS vtid, pid, granted
FROM pg_catalog.pg_locks l LEFT JOIN pg_catalog.pg_database db
ON db.oid = l.database WHERE (db.datname = 'udatapp_development' OR db.datname IS NULL)
AND NOT pid = pg_backend_pid();

 locktype | relation | mode | tid | vtid | pid | granted 
----------+----------+------+-----+------+-----+---------
(0 rows)

Raw

pg_locks.md

1 console:

udatapp_development=# begin;
BEGIN

udatapp_development=# select * from users;

2 console:

udatapp_development=# begin;
BEGIN

udatapp_development=# select * from users;

udatapp_development=# UPDATE users SET email = '[email protected]' WHERE id = 2;
UPDATE 1

3 console:

udatapp_development=# SELECT locktype, relation::regclass, mode, transactionid AS tid,
virtualtransaction AS vtid, pid, granted
FROM pg_catalog.pg_locks l LEFT JOIN pg_catalog.pg_database db
ON db.oid = l.database WHERE (db.datname = 'udatapp_development' OR db.datname IS NULL)
AND NOT pid = pg_backend_pid();

   locktype    |       relation       |       mode       |  tid   |  vtid   |   pid   | granted 
---------------+----------------------+------------------+--------+---------+---------+---------
 relation      | index_users_on_email | AccessShareLock  |        | 6/1332  | 3350240 | t
 relation      | index_users_on_email | RowExclusiveLock |        | 6/1332  | 3350240 | t
 relation      | users_pkey           | AccessShareLock  |        | 6/1332  | 3350240 | t
 relation      | users_pkey           | RowExclusiveLock |        | 6/1332  | 3350240 | t
 relation      | users                | AccessShareLock  |        | 6/1332  | 3350240 | t
 relation      | users                | RowExclusiveLock |        | 6/1332  | 3350240 | t
 virtualxid    |                      | ExclusiveLock    |        | 6/1332  | 3350240 | t
 relation      | index_users_on_email | AccessShareLock  |        | 5/20857 | 3350531 | t
 relation      | users_pkey           | AccessShareLock  |        | 5/20857 | 3350531 | t
 relation      | users                | AccessShareLock  |        | 5/20857 | 3350531 | t
 virtualxid    |                      | ExclusiveLock    |        | 5/20857 | 3350531 | t
 transactionid |                      | ExclusiveLock    | 818648 | 6/1332  | 3350240 | t
(12 rows)

1 console:


udatapp_development=# UPDATE users SET email = '[email protected]' WHERE id = 2;

...

3 console:

Look at ShareLock:

udatapp_development=# SELECT locktype, relation::regclass, mode, transactionid AS tid,
virtualtransaction AS vtid, pid, granted
FROM pg_catalog.pg_locks l LEFT JOIN pg_catalog.pg_database db
ON db.oid = l.database WHERE (db.datname = 'udatapp_development' OR db.datname IS NULL)
AND NOT pid = pg_backend_pid();
   locktype    |       relation       |        mode         |  tid   |  vtid   |   pid   | granted 
---------------+----------------------+---------------------+--------+---------+---------+---------
 relation      | index_users_on_email | AccessShareLock     |        | 6/1332  | 3350240 | t
 relation      | index_users_on_email | RowExclusiveLock    |        | 6/1332  | 3350240 | t
 relation      | users_pkey           | AccessShareLock     |        | 6/1332  | 3350240 | t
 relation      | users_pkey           | RowExclusiveLock    |        | 6/1332  | 3350240 | t
 relation      | users                | AccessShareLock     |        | 6/1332  | 3350240 | t
 relation      | users                | RowExclusiveLock    |        | 6/1332  | 3350240 | t
 virtualxid    |                      | ExclusiveLock       |        | 6/1332  | 3350240 | t
 relation      | index_users_on_email | AccessShareLock     |        | 5/20857 | 3350531 | t
 relation      | index_users_on_email | RowExclusiveLock    |        | 5/20857 | 3350531 | t
 relation      | users_pkey           | AccessShareLock     |        | 5/20857 | 3350531 | t
 relation      | users_pkey           | RowExclusiveLock    |        | 5/20857 | 3350531 | t
 relation      | users                | AccessShareLock     |        | 5/20857 | 3350531 | t
 relation      | users                | RowExclusiveLock    |        | 5/20857 | 3350531 | t
 virtualxid    |                      | ExclusiveLock       |        | 5/20857 | 3350531 | t
 transactionid |                      | ShareLock           | 818648 | 5/20857 | 3350531 | f
 transactionid |                      | ExclusiveLock       | 818648 | 6/1332  | 3350240 | t
 transactionid |                      | ExclusiveLock       | 818649 | 5/20857 | 3350531 | t
 tuple         | users                | AccessExclusiveLock |        | 5/20857 | 3350531 | t
(18 rows)

pg_stats_activity

3 console:

udatapp_development=# SELECT query, state, wait_event_type, wait_event, pid FROM pg_stat_activity
WHERE datname='udatapp_development' AND NOT (state='idle' OR pid=pg_backend_pid());

                            query                             |        state        | wait_event_type |  wait_event   |   pid   
--------------------------------------------------------------+---------------------+-----------------+---------------+---------
 UPDATE users SET email = '[email protected]' WHERE id = 2; | active              | Lock            | transactionid | 3350531
 UPDATE users SET email = '[email protected]' WHERE id = 2; | idle in transaction | Client          | ClientRead    | 3350240
(2 rows)

To look who is blocking to whom:

udatapp_development=# SELECT blockeda.pid AS blocked_pid, blockeda.query as blocked_query,
  blockinga.pid AS blocking_pid, blockinga.query as blocking_query
FROM pg_catalog.pg_locks blockedl
JOIN pg_stat_activity blockeda ON blockedl.pid = blockeda.pid
JOIN pg_catalog.pg_locks blockingl ON(blockingl.transactionid=blockedl.transactionid
  AND blockedl.pid != blockingl.pid)
JOIN pg_stat_activity blockinga ON blockingl.pid = blockinga.pid
WHERE NOT blockedl.granted AND blockinga.datname='udatapp_development';

 blocked_pid |                        blocked_query                         | blocking_pid |                        blocking_query                        
-------------+--------------------------------------------------------------+--------------+--------------------------------------------------------------
     3350531 | UPDATE users SET email = '[email protected]' WHERE id = 2; |      3350240 | UPDATE users SET email = '[email protected]' WHERE id = 2;
(1 row)

Rollback 2nd console transaction and commit 1st console transaction:

1 console:

udatapp_development=# commit;
COMMIT
udatapp_development=# select * from users;
 id |        email        |                       password_digest                        |         created_at         |         updated_at         
----+---------------------+--------------------------------------------------------------+----------------------------+----------------------------
  2 | [email protected] | $2a$12$.18O3pzTVIM1OhyFLLe/3uz7k8tlzAzkjBsY.bP7MWUD1Zf3Jvohm | 2022-03-22 00:08:13.714108 | 2022-03-22 00:08:13.714108
(1 row)

3 console:

udatapp_development=# SELECT locktype, relation::regclass, mode, transactionid AS tid,
virtualtransaction AS vtid, pid, granted
FROM pg_catalog.pg_locks l LEFT JOIN pg_catalog.pg_database db
ON db.oid = l.database WHERE (db.datname = 'udatapp_development' OR db.datname IS NULL)
AND NOT pid = pg_backend_pid();

 locktype | relation | mode | tid | vtid | pid | granted 
----------+----------+------+-----+------+-----+---------
(0 rows)

Raw

row_exclusive_lock.md

1st console:

udatapp_development=# begin; select * from users for update;

BEGIN
 id |             email             |                       password_digest                        |         created_at         |         updated_at         
----+-------------------------------+--------------------------------------------------------------+----------------------------+----------------------------
  2 | [email protected] | $2a$12$.18O3pzTVIM1OhyFLLe/3uz7k8tlzAzkjBsY.bP7MWUD1Zf3Jvohm | 2022-03-22 00:08:13.714108 | 2022-03-22 00:08:13.714108
(1 row)

2nd console:

udatapp_development=# UPDATE users SET email = '[email protected]' WHERE id = 2;

3rd console:

RowExclusiveLOck:

udatapp_development=# SELECT locktype, relation::regclass, mode, transactionid AS tid,
virtualtransaction AS vtid, pid, granted
FROM pg_catalog.pg_locks l LEFT JOIN pg_catalog.pg_database db
ON db.oid = l.database WHERE (db.datname = 'udatapp_development' OR db.datname IS NULL)
AND NOT pid = pg_backend_pid();

   locktype    |       relation       |        mode         |  tid   |  vtid   |   pid   | granted 
---------------+----------------------+---------------------+--------+---------+---------+---------
 relation      | index_users_on_email | RowExclusiveLock    |        | 6/1336  | 3350240 | t
 relation      | users_pkey           | RowExclusiveLock    |        | 6/1336  | 3350240 | t
 relation      | users                | RowExclusiveLock    |        | 6/1336  | 3350240 | t
 virtualxid    |                      | ExclusiveLock       |        | 6/1336  | 3350240 | t
 relation      | index_users_on_email | AccessShareLock     |        | 5/20861 | 3350531 | t
 relation      | index_users_on_email | RowShareLock        |        | 5/20861 | 3350531 | t
 relation      | users_pkey           | AccessShareLock     |        | 5/20861 | 3350531 | t
 relation      | users_pkey           | RowShareLock        |        | 5/20861 | 3350531 | t
 relation      | users                | AccessShareLock     |        | 5/20861 | 3350531 | t
 relation      | users                | RowShareLock        |        | 5/20861 | 3350531 | t
 virtualxid    |                      | ExclusiveLock       |        | 5/20861 | 3350531 | t
 tuple         | users                | AccessExclusiveLock |        | 6/1336  | 3350240 | t
 transactionid |                      | ExclusiveLock       | 818653 | 5/20861 | 3350531 | t
 transactionid |                      | ExclusiveLock       | 818654 | 6/1336  | 3350240 | t
 transactionid |                      | ShareLock           | 818653 | 6/1336  | 3350240 | f
(15 rows)

udatapp_development=# SELECT blockeda.pid AS blocked_pid,
udatapp_development-#   blockeda.query as blocked_query, blockinga.pid AS blocking_pid,
udatapp_development-#   blockinga.query as blocking_query
udatapp_development-# FROM pg_catalog.pg_locks blockedl
udatapp_development-# JOIN pg_stat_activity blockeda ON blockedl.pid = blockeda.pid
udatapp_development-# JOIN pg_catalog.pg_locks blockingl ON(
udatapp_development(#   blockingl.transactionid = blockedl.transactionid
udatapp_development(#   AND blockedl.pid != blockingl.pid)
udatapp_development-# JOIN pg_stat_activity blockinga ON blockingl.pid = blockinga.pid
udatapp_development-# WHERE NOT blockedl.granted
udatapp_development-# AND blockinga.datname='udatapp_development';

 blocked_pid |                                blocked_query                                | blocking_pid |    blocking_query    
-------------+-----------------------------------------------------------------------------+--------------+----------------------
     3350240 | UPDATE users SET email = '[email protected]' WHERE id = 2; |      3350531 | select * from users;
(1 row)

udatapp_development=# SELECT
udatapp_development-#   COALESCE(blockingl.relation::regclass::text,blockingl.locktype) as locked_item,
udatapp_development-#   blockeda.pid AS blocked_pid, blockeda.query as blocked_query,
udatapp_development-#   blockedl.mode as blocked_mode, blockinga.pid AS blocking_pid,
udatapp_development-#   blockinga.query as blocking_query, blockingl.mode as blocking_mode
udatapp_development-# FROM pg_catalog.pg_locks blockedl
udatapp_development-# JOIN pg_stat_activity blockeda ON blockedl.pid = blockeda.pid
udatapp_development-# JOIN pg_catalog.pg_locks blockingl ON(
udatapp_development(#   ( (blockingl.transactionid=blockedl.transactionid) OR
udatapp_development(#     (blockingl.relation=blockedl.relation AND blockingl.locktype=blockedl.locktype)
udatapp_development(#   ) AND blockedl.pid != blockingl.pid)
udatapp_development-# JOIN pg_stat_activity blockinga ON blockingl.pid = blockinga.pid
udatapp_development-# WHERE NOT blockedl.granted
udatapp_development-# AND blockinga.datname='udatapp_development';

  locked_item  | blocked_pid |                                blocked_query                                | blocked_mode | blocking_pid |    blocking_query    | blocking_mode 
---------------+-------------+-----------------------------------------------------------------------------+--------------+--------------+----------------------+---------------
 transactionid |     3350240 | UPDATE users SET email = '[email protected]' WHERE id = 2; | ShareLock    |      3350531 | select * from users; | ExclusiveLock
(1 row)

Create view to monitor:

udatapp_development=# CREATE VIEW lock_monitor AS(
udatapp_development(# SELECT
udatapp_development(#   COALESCE(blockingl.relation::regclass::text,blockingl.locktype) as locked_item,
udatapp_development(#   now() - blockeda.query_start AS waiting_duration, blockeda.pid AS blocked_pid,
udatapp_development(#   blockeda.query as blocked_query, blockedl.mode as blocked_mode,
udatapp_development(#   blockinga.pid AS blocking_pid, blockinga.query as blocking_query,
udatapp_development(#   blockingl.mode as blocking_mode
udatapp_development(# FROM pg_catalog.pg_locks blockedl
udatapp_development(# JOIN pg_stat_activity blockeda ON blockedl.pid = blockeda.pid
udatapp_development(# JOIN pg_catalog.pg_locks blockingl ON(
udatapp_development(#   ( (blockingl.transactionid=blockedl.transactionid) OR
udatapp_development(#   (blockingl.relation=blockedl.relation AND blockingl.locktype=blockedl.locktype)
udatapp_development(#   ) AND blockedl.pid != blockingl.pid)
udatapp_development(# JOIN pg_stat_activity blockinga ON blockingl.pid = blockinga.pid
udatapp_development(#   AND blockinga.datid = blockeda.datid
udatapp_development(# WHERE NOT blockedl.granted
udatapp_development(# AND blockinga.datname = current_database()
udatapp_development(# );

CREATE VIEW

Let's look at it:

udatapp_development=# SELECT * from lock_monitor;

  locked_item  | waiting_duration | blocked_pid |                                blocked_query                                | blocked_mode | blocking_pid |    blocking_query    | blocking_mode 
---------------+------------------+-------------+-----------------------------------------------------------------------------+--------------+--------------+----------------------+---------------
 transactionid | 00:02:47.441294  |     3350240 | UPDATE users SET email = '[email protected]' WHERE id = 2; | ShareLock    |      3350531 | select * from users; | ExclusiveLock
(1 row)


udatapp_development=# SELECT * from lock_monitor;

  locked_item  | waiting_duration | blocked_pid |                                blocked_query                                | blocked_mode | blocking_pid |    blocking_query    | blocking_mode 
---------------+------------------+-------------+-----------------------------------------------------------------------------+--------------+--------------+----------------------+---------------
 transactionid | 00:02:57.153558  |     3350240 | UPDATE users SET email = '[email protected]' WHERE id = 2; | ShareLock    |      3350531 | select * from users; | ExclusiveLock
(1 row)

Author

georgiybykov commented Mar 27, 2022

Explanation link to see more information.

psylone commented Mar 29, 2022

✅