Introduction:
For large SGA sizes, HugePages can give substantial benefits in virtual memory management. Without HugePages, the memory of the SGA is divided into 4K pages, which have to be managed by the Linux kernel. Using HugePages, the page size can be increased to anything between 2MB and 256MB, thereby reducing the total number of pages to be managed by the kernel and therefore reducing the amount of memory required to hold the page table in memory. In addition to these changes, the memory associated with HugePages cannot be swapped out, which forces the SGA to stay memory resident. The savings in memory and the effort of page management make HugePages pretty much mandatory for Oracle 11g systems running on x86-64 architectures.
Note: - Automatic Memory Management (AMM) is not compatible with Linux HugePages, so apart from ASM instances and small unimportant databases, you will probably have no need for AMM on a real database running on Linux. Instead, Automatic Shared Memory Management and Automatic PGA Management should be used as they are compatible with HugePages.
Disabling AMM from Oracle 11g Database:
From the above note, it clearly say AMM is not compatible with Linux HugePages, So for implementing HugePages firstly we need to disable AMM from running database. Please follow below steps to disable AMM from database:
1. Verify the AMM: Before disabling AMM, Please verify the memory target size in database:
SQL> sho parameter memory
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
hi_shared_memory_address integer 0
memory_max_target big integer 4G
memory_target big integer 4G
shared_memory_address integer 0
From the above output, we came to know the AMM is enabled and memory_target is 4GB. So we need to disable it first.
2. Create PFILE from SPFILE.
SQL> create pfile from spfile;
File created.
3. Edit Pfile: In Pfile we have to add and remove some parameter. Please find below:
Parameters which need to be remove:
a. memory_target=4294967296
b. memory_max_target=4294967296
c. log_archive_start=TRUE
d. shared_io_pool_size=0
Parameters which need to be add:
a. sga_max_size=4294967296
b. sga_target=2801795072
c. pga_aggregate_target=1073741824
4. After preparing the PFILE, bring down the database and start using newly created pfile:
SQL> shut immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup pfile=$ORACLE_HOME/dbs/initTEST1.ora
ORACLE instance started.
Total System Global Area 3206836224 bytes
Fixed Size 2217632 bytes
Variable Size 2147486048 bytes
Database Buffers 1040187392 bytes
Redo Buffers 16945152 bytes
Database mounted.
Database opened.
5. After database get started, validate that AMM has disabled or not:
SQL> sho parameter memory
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
hi_shared_memory_address integer 0
memory_max_target big integer 0
memory_target big integer 0
shared_memory_address integer 0
From above output, we are sure now that AMM has been disable, Now we can further towards the HugePages implementation. But before going to implement HugePages it will be better to switch to SPFILE from PFILE.
6. Creating SPFILE from PFILE and bring up the database using SPFILE and verifying once again AMM for safer side:
SQL> create spfile from pfile='/t01/app/oracle/product/11.2.0/server/dbs/initTEST1.ora';
File created.
SQL> shut immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup
ORACLE instance started.
Total System Global Area 3206836224 bytes
Fixed Size 2217632 bytes
Variable Size 2147486048 bytes
Database Buffers 1040187392 bytes
Redo Buffers 16945152 bytes
Database mounted.
Database opened.
SQL> show parameter target
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
memory_max_target big integer 0
memory_target big integer 0
Configuring HugePages:
1. Run the following command to determine the current HugePage usage. The default HugePage size is 2MB on Oracle Linux 6.x and as you can see from the output below, by default no HugePages are defined.
[oracle@nodedb1 dbs]$ grep Huge /proc/meminfo
AnonHugePages: 8192 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
2. Create a file called "hugepages_setting.sh" with the following contents.
[oracle@nodedb1 Hugepages]$ cat > hugepages_setting.sh
#!/bin/bash
#
# hugepages_settings.sh
#
# Linux bash script to compute values for the
# recommended HugePages/HugeTLB configuration
#
# Note: This script does calculation for all shared memory
# segments available when the script is run, no matter it
# is an Oracle RDBMS shared memory segment or not.
# Check for the kernel version
KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`
# Find out the HugePage size
HPG_SZ=`grep Hugepagesize /proc/meminfo | awk {'print $2'}`
# Start from 1 pages to be on the safe side and guarantee 1 free HugePage
NUM_PG=1
# Cumulative number of pages required to handle the running shared memory segments
for SEG_BYTES in `ipcs -m | awk {'print $5'} | grep "[0-9][0-9]*"`
do
MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`
if [ $MIN_PG -gt 0 ]; then
NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`
fi
done
# Finish with results
case $KERN in
'2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;
echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;
'2.6') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
*) echo "Unrecognized kernel version $KERN. Exiting." ;;
esac
# End
3. Make the file executable.
[oracle@nodedb1 Hugepages]$ chmod u+x hugepages_setting.sh
4. Run the script and make a note of the recommended "vm.nr_hugepages" value.
Note: Make sure all the Oracle services are running as normal on the server.
[oracle@nodedb1 Hugepages]$ ./hugepages_setting.sh
Recommended setting: vm.nr_hugepages = 1539
5. Edit the "/etc/sysctl.conf" file as the "root" user, adding the following entry, adjusted based on your output from the script. You should set the value greater than or equal to the value displayed by the script. You only need 1 or 2 spare pages.
Recommended setting: vm.nr_hugepages = 1541
6. Run the following command as the "root" user.
root@nodedb1:~ # sysctl –p
7. You can now see the HugePages have been created, but are currently not being used.
[oracle@nodedb1 ~]$ grep Huge /proc/meminfo
AnonHugePages: 10240 kB
HugePages_Total: 1541
HugePages_Free: 1541
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
8. Add the following entries into the "/etc/security/limits.conf" script, where the setting is the size of the HugePages allocation in KB (HugePages * Hugepagesize). In this case the value is 1541*2048=3155968.
* soft memlock 3155968
* hard memlock 3155968
9. Check the MEMORY_TARGET parameters are not set for the database and SGA_TARGET and PGA_AGGREGATE_TARGET parameters are being used instead.
SQL> show parameter target
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
archive_lag_target integer 0
db_flashback_retention_target integer 1440
fast_start_io_target integer 0
fast_start_mttr_target integer 0
memory_max_target big integer 0
memory_target big integer 0
parallel_servers_target integer 128
pga_aggregate_target big integer 1424M
sga_target big integer 2672M
10. Restart the server and restart the database services as required.
11. Check the HugePages information again.
[oracle@nodedb1 ~]$ grep Huge /proc/meminfo
AnonHugePages: 0 kB
HugePages_Total: 1541
HugePages_Free: 1168
HugePages_Rsvd: 1164
HugePages_Surp: 0
Hugepagesize: 2048 kB
You can see the HugePages are now being used.
For large SGA sizes, HugePages can give substantial benefits in virtual memory management. Without HugePages, the memory of the SGA is divided into 4K pages, which have to be managed by the Linux kernel. Using HugePages, the page size can be increased to anything between 2MB and 256MB, thereby reducing the total number of pages to be managed by the kernel and therefore reducing the amount of memory required to hold the page table in memory. In addition to these changes, the memory associated with HugePages cannot be swapped out, which forces the SGA to stay memory resident. The savings in memory and the effort of page management make HugePages pretty much mandatory for Oracle 11g systems running on x86-64 architectures.
Note: - Automatic Memory Management (AMM) is not compatible with Linux HugePages, so apart from ASM instances and small unimportant databases, you will probably have no need for AMM on a real database running on Linux. Instead, Automatic Shared Memory Management and Automatic PGA Management should be used as they are compatible with HugePages.
Disabling AMM from Oracle 11g Database:
From the above note, it clearly say AMM is not compatible with Linux HugePages, So for implementing HugePages firstly we need to disable AMM from running database. Please follow below steps to disable AMM from database:
1. Verify the AMM: Before disabling AMM, Please verify the memory target size in database:
SQL> sho parameter memory
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
hi_shared_memory_address integer 0
memory_max_target big integer 4G
memory_target big integer 4G
shared_memory_address integer 0
From the above output, we came to know the AMM is enabled and memory_target is 4GB. So we need to disable it first.
2. Create PFILE from SPFILE.
SQL> create pfile from spfile;
File created.
3. Edit Pfile: In Pfile we have to add and remove some parameter. Please find below:
Parameters which need to be remove:
a. memory_target=4294967296
b. memory_max_target=4294967296
c. log_archive_start=TRUE
d. shared_io_pool_size=0
Parameters which need to be add:
a. sga_max_size=4294967296
b. sga_target=2801795072
c. pga_aggregate_target=1073741824
4. After preparing the PFILE, bring down the database and start using newly created pfile:
SQL> shut immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup pfile=$ORACLE_HOME/dbs/initTEST1.ora
ORACLE instance started.
Total System Global Area 3206836224 bytes
Fixed Size 2217632 bytes
Variable Size 2147486048 bytes
Database Buffers 1040187392 bytes
Redo Buffers 16945152 bytes
Database mounted.
Database opened.
5. After database get started, validate that AMM has disabled or not:
SQL> sho parameter memory
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
hi_shared_memory_address integer 0
memory_max_target big integer 0
memory_target big integer 0
shared_memory_address integer 0
From above output, we are sure now that AMM has been disable, Now we can further towards the HugePages implementation. But before going to implement HugePages it will be better to switch to SPFILE from PFILE.
6. Creating SPFILE from PFILE and bring up the database using SPFILE and verifying once again AMM for safer side:
SQL> create spfile from pfile='/t01/app/oracle/product/11.2.0/server/dbs/initTEST1.ora';
File created.
SQL> shut immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup
ORACLE instance started.
Total System Global Area 3206836224 bytes
Fixed Size 2217632 bytes
Variable Size 2147486048 bytes
Database Buffers 1040187392 bytes
Redo Buffers 16945152 bytes
Database mounted.
Database opened.
SQL> show parameter target
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
memory_max_target big integer 0
memory_target big integer 0
Configuring HugePages:
1. Run the following command to determine the current HugePage usage. The default HugePage size is 2MB on Oracle Linux 6.x and as you can see from the output below, by default no HugePages are defined.
[oracle@nodedb1 dbs]$ grep Huge /proc/meminfo
AnonHugePages: 8192 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
2. Create a file called "hugepages_setting.sh" with the following contents.
[oracle@nodedb1 Hugepages]$ cat > hugepages_setting.sh
#!/bin/bash
#
# hugepages_settings.sh
#
# Linux bash script to compute values for the
# recommended HugePages/HugeTLB configuration
#
# Note: This script does calculation for all shared memory
# segments available when the script is run, no matter it
# is an Oracle RDBMS shared memory segment or not.
# Check for the kernel version
KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`
# Find out the HugePage size
HPG_SZ=`grep Hugepagesize /proc/meminfo | awk {'print $2'}`
# Start from 1 pages to be on the safe side and guarantee 1 free HugePage
NUM_PG=1
# Cumulative number of pages required to handle the running shared memory segments
for SEG_BYTES in `ipcs -m | awk {'print $5'} | grep "[0-9][0-9]*"`
do
MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`
if [ $MIN_PG -gt 0 ]; then
NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`
fi
done
# Finish with results
case $KERN in
'2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;
echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;
'2.6') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
*) echo "Unrecognized kernel version $KERN. Exiting." ;;
esac
# End
3. Make the file executable.
[oracle@nodedb1 Hugepages]$ chmod u+x hugepages_setting.sh
4. Run the script and make a note of the recommended "vm.nr_hugepages" value.
Note: Make sure all the Oracle services are running as normal on the server.
[oracle@nodedb1 Hugepages]$ ./hugepages_setting.sh
Recommended setting: vm.nr_hugepages = 1539
5. Edit the "/etc/sysctl.conf" file as the "root" user, adding the following entry, adjusted based on your output from the script. You should set the value greater than or equal to the value displayed by the script. You only need 1 or 2 spare pages.
Recommended setting: vm.nr_hugepages = 1541
6. Run the following command as the "root" user.
root@nodedb1:~ # sysctl –p
7. You can now see the HugePages have been created, but are currently not being used.
[oracle@nodedb1 ~]$ grep Huge /proc/meminfo
AnonHugePages: 10240 kB
HugePages_Total: 1541
HugePages_Free: 1541
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
8. Add the following entries into the "/etc/security/limits.conf" script, where the setting is the size of the HugePages allocation in KB (HugePages * Hugepagesize). In this case the value is 1541*2048=3155968.
* soft memlock 3155968
* hard memlock 3155968
9. Check the MEMORY_TARGET parameters are not set for the database and SGA_TARGET and PGA_AGGREGATE_TARGET parameters are being used instead.
SQL> show parameter target
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
archive_lag_target integer 0
db_flashback_retention_target integer 1440
fast_start_io_target integer 0
fast_start_mttr_target integer 0
memory_max_target big integer 0
memory_target big integer 0
parallel_servers_target integer 128
pga_aggregate_target big integer 1424M
sga_target big integer 2672M
10. Restart the server and restart the database services as required.
11. Check the HugePages information again.
[oracle@nodedb1 ~]$ grep Huge /proc/meminfo
AnonHugePages: 0 kB
HugePages_Total: 1541
HugePages_Free: 1168
HugePages_Rsvd: 1164
HugePages_Surp: 0
Hugepagesize: 2048 kB
You can see the HugePages are now being used.
No comments:
Post a Comment