You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
Currently, libcudf supports upto 32-bit integers. Hence, the NDS-H data generator which is built using libcudf can only generate upto 2,147,483,647 rows before it hits the row limit. For NDS-H, this translates to a scale factor of ~350 (The lineitem table, which is the largest NDS-H table, has ~6M rows, so the largest lineitem table that can be generated is of scale factor 350). But, when we tried running the NDS-H data generator with scale factor > 150, it stalled indefinitely. The highest scale factor that it was able to successfully generate was 150. This was observed both with un-managed and managed memory, and the execution stalled even though there was enough memory left.
Steps/Code to reproduce bug
Run the data generator with scale factor > 150
Expected behavior
The NDS-H data generator should be able to generate upto scale factor ~350.
Environment overview (please complete the following information)
Environment location: NVIDIA H100 and NVIDIA GH200
Method of cuDF install: Built from source, branch-24.10
The text was updated successfully, but these errors were encountered:
Describe the bug
Currently,
libcudf
supports upto 32-bit integers. Hence, the NDS-H data generator which is built usinglibcudf
can only generate upto 2,147,483,647 rows before it hits the row limit. For NDS-H, this translates to a scale factor of ~350 (Thelineitem
table, which is the largest NDS-H table, has ~6M rows, so the largestlineitem
table that can be generated is of scale factor 350). But, when we tried running the NDS-H data generator with scale factor > 150, it stalled indefinitely. The highest scale factor that it was able to successfully generate was 150. This was observed both with un-managed and managed memory, and the execution stalled even though there was enough memory left.Steps/Code to reproduce bug
Run the data generator with scale factor > 150
Expected behavior
The NDS-H data generator should be able to generate upto scale factor ~350.
Environment overview (please complete the following information)
The text was updated successfully, but these errors were encountered: