Geomatips

S'abonner à flux Geomatips
Follow me on twitter : @GISCAN_LTD https://twitter.com/GISCAN_LTD or on GISCAN.COMSylvain POULAINhttp://www.blogger.com/profile/01217650890209665453noreply@blogger.comBlogger14125
Mis à jour : il y a 1 heure 44 min

My GIS performance impact of Meltdown and Spectre patches

mar, 01/09/2018 - 19:34
When I saw performance regression few days ago of Meltdown and Spectre patch from cpu conception problem, I would like to perform some basic test to understand if it could slow down my process.

For those who don't know, here : a postgres article about performance regression : https://www.postgresql.org/message-id/20180102222354.qikjmf7dvnjgbkxe@alap3.anarazel.de 
Since some users finished to make benchmark with Meltdown and Spectre patch, I will not test it on my computer.

How to test  ?
First, I tested it on Archlinux based distribution named Manjaro Linux.


To know if patch is applied to my kernel, I just run this command :

$ cat /proc/cpuinfo | grep cpu_insecureWhich reports this :

bugs        : cpu_insecure
bugs        : cpu_insecure
bugs        : cpu_insecure
bugs        : cpu_insecure
bugs        : cpu_insecure
bugs        : cpu_insecure
bugs        : cpu_insecure
bugs        : cpu_insecureWhen showing cpu_insecure, you know that KPTI patch is enable.

Then there is a specific C code to test your cpu : 

    #include
    #include
    #include
    #include
    // gcc -Wall -O3 -o getpid getpid.c
    int main(void)
    {
    long value;
    clock_t begin = clock();  
    for (int i = 0; i < 100000000; i++)
     value = syscall(SYS_getpid);
    clock_t end = clock();
    double time_spent = (double)(end - begin) / CLOCKS_PER_SEC;
    printf("%d\n", (int) value);
    printf("%f\n", time_spent);
    return 0;
    }


Write these lines to pti.c file, build it with :

gcc -Wall -O3 -o pti pti.c
and finally run it with
./ptiFor my cpu aka Intel i7 4790k, switching to patched kernel, my computer slow down a lot from this generic test. When upgrading to pacthed kernel, you can still disable patch with kernel grub option pti=off. Next tests will be shown with pti=on and pti=off

With pti=off and generic test, my results could be shown below :


1211
3.571401 seconds With pti=on and generic, we could watch a several regression :
1208
13.998673 seconds Now to know when rebooting my system, I know if patch is apply or not and before making tests, I will build and run this C code.
Simple GIS test : 
From years now and unfortunately, I fight with ECW files. I have some big ecw which I found far away better in performance in comparison to multiple files like 36GB.
But for this test I will use a moderate ECW raster which is an orthophotography of Mauritius Island with 15cm resolution pixel and size of this file is 15 GB and cover 56 x 67 kilometers. Sample overview :

For those are curious here is in detail :

$ gdalinfo ALL.ecw
Driver: ECW/ERDAS Compressed Wavelets (SDK 3.x)
Files: ALL.ecw
       ALL.ecw.aux.xml
Size is 372000, 450000
Coordinate System is:
PROJCS["SUTM40",
    GEOGCS["WGS 84",
        DATUM["WGS_1984",
            SPHEROID["WGS 84",6378137,298.257223563,
                AUTHORITY["EPSG","7030"]],
            TOWGS84[0,0,0,0,0,0,0],
            AUTHORITY["EPSG","6326"]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9108"]],
        AXIS["Lat",NORTH],
        AXIS["Long",EAST],
        AUTHORITY["EPSG","4326"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",57],
    PARAMETER["scale_factor",0.9996],
    PARAMETER["false_easting",500000],
    PARAMETER["false_northing",10000000],
    UNIT["Meter",1]]
Origin = (531000.000000000000000,7796700.000000000000000)
Pixel Size = (0.149999999999993,-0.149999999999993)
Metadata:
  COLORSPACE=RGB
  COMPRESSION_RATE_TARGET=4
  VERSION=2
Corner Coordinates:
Upper Left  (  531000.000, 7796700.000) ( 57d17'46.37"E, 19d55'32.96"S)
Lower Left  (  531000.000, 7729200.000) ( 57d17'50.53"E, 20d32' 8.78"S)
Upper Right (  586800.000, 7796700.000) ( 57d49'45.72"E, 19d55'26.88"S)
Lower Right (  586800.000, 7729200.000) ( 57d49'57.39"E, 20d32' 2.50"S)
Center      (  558900.000, 7762950.000) ( 57d33'49.99"E, 20d13'48.53"S)
Band 1 Block=256x256 Type=Byte, ColorInterp=Red
  Min=0.000 Max=255.000
  Minimum=0.000, Maximum=255.000, Mean=59.587, StdDev=56.411
  Overviews: 186000x225000, 93000x112500, 46500x56250, 23250x28125, 11625x14062, 5812x7031, 2906x3515, 1453x1757, 726x878, 363x439, 181x219
  Metadata:
    STATISTICS_MAXIMUM=255
    STATISTICS_MEAN=59.586901452342
    STATISTICS_MINIMUM=0
    STATISTICS_STDDEV=56.410809243179
Band 2 Block=256x256 Type=Byte, ColorInterp=Green
  Min=0.000 Max=255.000
  Minimum=0.000, Maximum=255.000, Mean=57.965, StdDev=53.573
  Overviews: 186000x225000, 93000x112500, 46500x56250, 23250x28125, 11625x14062, 5812x7031, 2906x3515, 1453x1757, 726x878, 363x439, 181x219
  Metadata:
    STATISTICS_MAXIMUM=255
    STATISTICS_MEAN=57.964855468501
    STATISTICS_MINIMUM=0
    STATISTICS_STDDEV=53.572679192972
Band 3 Block=256x256 Type=Byte, ColorInterp=Blue
  Min=0.000 Max=255.000
  Minimum=0.000, Maximum=255.000, Mean=47.609, StdDev=46.083
  Overviews: 186000x225000, 93000x112500, 46500x56250, 23250x28125, 11625x14062, 5812x7031, 2906x3515, 1453x1757, 726x878, 363x439, 181x219
  Metadata:
    STATISTICS_MAXIMUM=255
    STATISTICS_MEAN=47.608960074074
    STATISTICS_MINIMUM=0
    STATISTICS_STDDEV=46.082615455709


Idea is to create a pretty well compress geotiff (and a good way to switch from ECW to geotiff since Paul Ramsey make a great tutorial about it and it's now embeded in Qgis raster menu ! ) to understand if these kernel patches have influences over a standard GIS tranformation.
I use gdal_translate with time command in a bash terminal to benchmark it. Command line is like that :

$ time gdal_translate -co BIGTIFF=YES -co COMPRESS=JPEG -co PHOTOMETRIC=YCBCR -co TILED=YES -co NUM_THREADS=8 in.ecw out.tif With pti=off I've got this result : 


Input file size is 372000, 450000
0...10...20...30...40...50...60...70...80...90...100 - done.

real 113m36,193s
user 106m22,559s
sys 0m34,030s And with pti=on this :
Input file size is 372000, 450000
0...10...20...30...40...50...60...70...80...90...100 - done.

real 117m4,998s
user 107m28,128s
sys 0m44,773s
Results shiow +/- no difference which demonstrate there is no impact regression on a Gdal GIS usage. Time for this process is big due to usage of JPEG compression which is only single core in gdal. Test is perform on external usb 3.0 2.5" hard drive.I made some others tests with gdalwrap and multithreading option and also there is no significant differences ...CPU / GPU test with blender and generic blenchmark : 
Finally, I use my computer to render 3D files sometimes and I would like to test CPU and GPU in this test. May be not related but cpu manage (with a small amount) gpu during gpu computation.

With pti on or off I had +/- same results : 45/47s with dual GPU and 2min10 / 2min11




So applying and using patch is recommanded since they publish how to use this leaks but we clearly show that they want to sell new platforms with a great new performances (in fact same as before without KPTI patches).
Happy proofing and selling thing but in my case it doesn't matter except with my postgres / postgis database. Does postgres tests made by others with pgbench really perform impact of GIS usage ?!

Catégories: Technique