skip to Main Content

I have a table users (username, email, validts, confirmed.
I want add 1 billion unique rows

I tried to do this but it only adds 16 unique values

INSERT INTO users (username, email, validts, confirmed)
SELECT 
    CONCAT('user', n) AS username,
    CONCAT('user', n, '@example.com') AS email,
    UNIX_TIMESTAMP() + 2592000 AS validts,
    0 AS confirmed
FROM 
    (SELECT @n := @n + 1 AS n FROM (SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3) t1, (SELECT 0 UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 10000) t2, (SELECT @n := 0) t3) numbers
LIMIT 1000000;

2

Answers


  1. in MySQL 5.7 this would work

    delimiter //
    
    CREATE PROCEDURE populate (IN num INT)
           BEGIN
              WHILE num > 0 DO
                START TRANSACTION;
                INSERT INTO users (username, email, validts, confirmed)
                SELECT 
                    CONCAT('user', cast(num as CHAR)),
                    CONCAT('user', cast(num as CHAR), '@example.com'),
                    FROM_UNIXTIME(UNIX_TIMESTAMP() + 2592000),
                    0;
                COMMIT;
                SET num = num - 1;
              END WHILE;
           END//
    
    delimiter ;
    
    call populate(1000000000);
    

    but this will take very long time. you may try to move START TRANSACTION before WHILE loop, so MySQL will do one big transaction instead of 1 billion of small ones. That will speed it up a little bit, but in case of failure or Ctrl-C it will rollback all the inserted lines. Also disabling indexes on the table and disabling binlog would help a little bit. Beware, disabling binlog will break replication, if it takes place.
    In more recent version you might be able to achieve this with recursive CTE. You generate CTE recursively until reaching 1 billion, and then insert into table from it. I think this should be faster.

    Login or Signup to reply.
  2. You might try this technique to generate Ns (and you can add other columns as you need):

    create table dmy2 as 
    select ((((T1.V)*10+T2.V)*10+T3.V)*10+T4.V)*10+T5.V as N
    from       (values ROW (0),ROW (1),ROW (2),ROW (3),ROW (4),ROW (5),ROW (6),ROW (7),ROW (8),ROW (9)) T1(V)
    cross join (values ROW (0),ROW (1),ROW (2),ROW (3),ROW (4),ROW (5),ROW (6),ROW (7),ROW (8),ROW (9)) T2(V)
    cross join (values ROW (0),ROW (1),ROW (2),ROW (3),ROW (4),ROW (5),ROW (6),ROW (7),ROW (8),ROW (9)) T3(V)
    cross join (values ROW (0),ROW (1),ROW (2),ROW (3),ROW (4),ROW (5),ROW (6),ROW (7),ROW (8),ROW (9)) T4(V)
    cross join (values ROW (0),ROW (1),ROW (2),ROW (3),ROW (4),ROW (5),ROW (6),ROW (7),ROW (8),ROW (9)) T5(V)
    

    You need to continue with the pattern to T9; I don’t have any facility to test how long it takes, and you may need to perform some type-casting if the value of N overflows.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search