When fsync is done through checkpoint, previous f2fs missed to clear append
and update flag. This patch fixes to clear them.

This was originally catched by Changman Lee before.

Signed-off-by: Changman Lee <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

If inode has inline_data, it should report -ENOENT when accessing out-of-bound
region.
This is used by f2fs_fiemap which treats -ENOENT with no error.

Signed-off-by: Jaegeuk Kim <[email protected]>

The f2fs has been shipped on many smartphone devices during a couple of years.
So, it is worth to relocate Kconfig into main page from misc filesystems for
developers to choose it more easily.

Signed-off-by: Jaegeuk Kim <[email protected]>

extent tree/node slab cache is created during f2fs insmod,
how, it isn't destroyed during f2fs rmmod, this patch fix
it by destroy extent tree/node slab cache once rmmod f2fs.

Signed-off-by: Wanpeng Li <[email protected]>
Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

In __set_free we will check whether all segment are free in one section
when free one segment, in order to set section to free status. But the
searching region of segmap is from start segno to last segno of main
area, it's not necessary. So let's just only check all segment bitmap
of target section.

Signed-off-by: Wanpeng Li <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

The function 'find_in_inline_dir()' contain 'res_page'
as an argument. So, we should initiaize 'res_page' before
this function.

Signed-off-by: Yuan Zhong <[email protected]>
Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

Remove the unnecessary condition judgment, because
'max_slots' has been initialized to '0' at the beginging
of the function, as following:
if (max_slots)
       *max_slots = 0;

Signed-off-by: Yuan Zhong <[email protected]>
Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

Through each macro, we can read the meaning easily.

Signed-off-by: Changman Lee <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

nm_i->nat_tree_lock is used to sync both the operations of nat entry
cache tree and nat set cache tree, however, it isn't held when flush
nat entries during checkpoint which lead to potential race, this patch
fix it by holding the lock when gang lookup nat set cache and delete
item from nat set cache.

Signed-off-by: Wanpeng Li <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

When lookuping for creating, we will try to record the level of current dentry
hash table if current dentry has enough contiguous slots for storing name of new
file which will be created later, this can save our lookup time when add a link
into parent dir.

But currently in find_target_dentry, our current length of contiguous free slots
is not calculated correctly. This make us leaving some holes in dentry block
occasionally, it wastes our space of dentry block.

Let's refactor the lookup flow for max slots as following to fix this issue:
a) increase max_len if current slot is free;
b) update max_slots with max_len if max_len is larger than max_slots;
c) reset max_len to zero if current slot is not free.

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

Our f2fs_acl_create is copied and modified from posix_acl_create to avoid
deadlock bug when inline_dentry feature is enabled.

Now, we got reference leaks in posix_acl_create, and this has been fixed in
commit fed0b58 ("posix_acl: fix reference leaks in posix_acl_create")
by Omar Sandoval.
https://lkml.org/lkml/2015/2/9/5

Let's fix this issue in f2fs_acl_create too.

Signed-off-by: Chao Yu <[email protected]>
Reviewed-by: Changman Lee <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

Previously if inode is with inline data, we will try to invalid partial inline
data in page #0 when we truncate size of inode in truncate_partial_data_page().
And then we set page #0 to dirty, after this we can synchronize inode page with
page #0 at ->writepage().

But sometimes we will fail to operate page #0 in truncate_partial_data_page()
due to below reason:
a) if offset is zero, we will skip setting page #0 to dirty.
b) if page #0 is not uptodate, we will fail to update it as it has no mapping
data.

So with following operations, we will meet recent data which should be
truncated.

1.write inline data to file
2.sync first data page to inode page
3.truncate file size to 0
4.truncate file size to max_inline_size
5.echo 1 > /proc/sys/vm/drop_caches
6.read file --> meet original inline data which is remained in inode page.

This patch renames truncate_inline_data() to truncate_inline_inode() for code
readability, then use truncate_inline_inode() to truncate inline data in inode
page in truncate_blocks() and truncate page #0 in truncate_partial_data_page()
for fixing.

v2:
 o truncate partially #0 page in truncate_partial_data_page to avoid keeping
   old data in #0 page.

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

In __allocate_data_blocks, we should check current blkaddr which is located at
ofs_in_node of dnode page instead of checking first blkaddr all the time.
Otherwise we can only allocate one blkaddr in each dnode page. Fix it.

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

In the following call stack, f2fs changes the bitmap for dirty segments and # of
dirty sentries without grabbing sit_i->sentry_lock.
This can result in mismatch on bitmap and # of dirty sentries, since if there
are some direct_io operations.

In allocate_data_block,
 - __allocate_new_segments
  - mutex_lock(&curseg->curseg_mutex);
  - s_ops->allocate_segment
   - new_curseg/change_curseg
    - reset_curseg
     - __set_sit_entry_type
      - __mark_sit_entry_dirty
       - set_bit(dirty_sentries_bitmap)
       - dirty_sentries++;

Signed-off-by: Jaegeuk Kim <[email protected]>

This patch tries to set SBI_NEED_FSCK flag into sbi only when we fail to recover
in fill_super, so we could skip fscking image when we fail to fill super for
other reason.

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

This patch modifies to call set_buffer_new, if new blocks are allocated.

Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

Previously, f2fs_write_data_pages has a mutex, sbi->writepages, to serialize
data writes to maximize write bandwidth, while sacrificing multi-threads
performance.
Practically, however, multi-threads environment is much more important for
users. So this patch tries to remove the mutex.

Signed-off-by: Jaegeuk Kim <[email protected]>

This patch removes wrong f2fs_bug_on in truncate_inline_inode.

When there is no space, it can happen a corner case where i_isze is over
MAX_INLINE_SIZE while its inode is still inline_data.

The scenario is
 1. write small data into file #A.
 2. fill the whole partition to 100%.
 3. truncate 4096 on file #A.
 4. write data at 8192 offset.
  --> f2fs_write_begin
    -> -ENOSPC = f2fs_convert_inline_page
    -> f2fs_write_failed
      -> truncate_blocks
        -> truncate_inline_inode
	  BUG_ON, since i_size is 4096.

Reviewed-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

This patch is to avoid some punch_hole overhead when releasing volatile data.
If volatile data was not written yet, we just can make the first page as zero.

Signed-off-by: Jaegeuk Kim <[email protected]>

Signed-off-by: Jaegeuk Kim <[email protected]>

Fast symlink can utilize inline data flow to avoid using any
i_addr region, since we need to handle many cases such as
truncation, roll-forward recovery, and fsck/dump tools.

Signed-off-by: Wanpeng Li <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

Split __set_data_blkaddr from f2fs_update_extent_cache for readability.

Additionally rename __set_data_blkaddr to set_data_blkaddr for exporting.

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

This patch introduces __{find,grab}_extent_tree for reusing by following
patches.

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

With normal extent info cache, we records largest extent mapping between logical
block and physical block into extent info, and we persist extent info in on-disk
inode.

When we enable extent tree cache, if extent info of on-disk inode is exist, and
the extent is not a small fragmented mapping extent. We'd better to load the
extent info into extent tree cache when inode is loaded. By this way we can have
more chance to hit extent tree cache rather than taking more time to read dnode
page for block address.

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

This patch tries to preserve last extent info in extent tree cache into on-disk
inode, so this can help us to reuse the last extent info next time for
performance.

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

Enable inline_data feature by default since it brings us better
performance and space utilization and now has already stable.
Add another option noinline_data to disable it during mount.

Suggested-by: Jaegeuk Kim <[email protected]>
Suggested-by: Chao Yu <[email protected]>
Signed-off-by: Wanpeng Li <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

Normally, due to DIO_SKIP_HOLES flag is set by default, blockdev_direct_IO in
f2fs_direct_IO tries to skip DIO in holes when writing inside i_size, this
makes us falling back to buffered IO which shows lower performance.

So in commit 59b802e ("f2fs: allocate data blocks in advance for
f2fs_direct_IO"), we improve perfromance by allocating data blocks in advance
if we meet holes no matter in i_size or not, since with it we can avoid falling
back to buffered IO.

But we forget to consider for unwritten fallocated block in this commit.
This patch tries to fix it for fallocate case, this helps to improve
performance.

Test result:
Storage info: sandisk ultra 64G micro sd card.

touch /mnt/f2fs/file
truncate -s 67108864 /mnt/f2fs/file
fallocate -o 0 -l 67108864 /mnt/f2fs/file
time dd if=/dev/zero of=/mnt/f2fs/file bs=1M count=64 conv=notrunc oflag=direct

Time before applying the patch:
67108864 bytes (67 MB) copied, 36.16 s, 1.9 MB/s
real    0m36.162s
user    0m0.000s
sys     0m0.180s

Time after applying the patch:
67108864 bytes (67 MB) copied, 27.7776 s, 2.4 MB/s
real    0m27.780s
user    0m0.000s
sys     0m0.036s

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

We will encounter oops by executing below command.
getfattr -n system.advise /mnt/f2fs/file
Killed

message log:
BUG: unable to handle kernel NULL pointer dereference at   (null)
IP: [<f8b54d69>] f2fs_xattr_advise_get+0x29/0x40 [f2fs]
*pdpt = 00000000319b7001 *pde = 0000000000000000
Oops: 0002 [#1] SMP
Modules linked in: f2fs(O) snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm snd_seq_midi snd_seq_midi_event snd_rawmidi snd_seq joydev
snd_seq_device snd_timer bnep snd rfcomm microcode bluetooth soundcore i2c_piix4 mac_hid serio_raw parport_pc ppdev lp parport
binfmt_misc hid_generic psmouse usbhid hid e1000 [last unloaded: f2fs]
CPU: 3 PID: 3134 Comm: getfattr Tainted: G           O    4.0.0-rc1 #6
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: f3a71b60 ti: f19a6000 task.ti: f19a6000
EIP: 0060:[<f8b54d69>] EFLAGS: 00010246 CPU: 3
EIP is at f2fs_xattr_advise_get+0x29/0x40 [f2fs]
EAX: 00000000 EBX: f19a7e71 ECX: 00000000 EDX: f8b5b467
ESI: 00000000 EDI: f2008570 EBP: f19a7e14 ESP: f19a7e08
 DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
CR0: 80050033 CR2: 00000000 CR3: 319b8000 CR4: 000007f0
Stack:
 f8b5a634 c0cbb580 00000000 f19a7e34 c1193850 00000000 00000007 f19a7e71
 f19a7e64 c0cbb580 c1193810 f19a7e50 c1193c00 00000000 00000000 00000000
 c0cbb580 00000000 f19a7f70 c1194097 00000000 00000000 00000000 74737973
Call Trace:
 [<c1193850>] generic_getxattr+0x40/0x50
 [<c1193810>] ? xattr_resolve_name+0x80/0x80
 [<c1193c00>] vfs_getxattr+0x70/0xa0
 [<c1194097>] getxattr+0x87/0x190
 [<c11801d7>] ? path_lookupat+0x57/0x5f0
 [<c11819d2>] ? putname+0x32/0x50
 [<c116653a>] ? kmem_cache_alloc+0x2a/0x130
 [<c11819d2>] ? putname+0x32/0x50
 [<c11819d2>] ? putname+0x32/0x50
 [<c11819d2>] ? putname+0x32/0x50
 [<c11827f9>] ? user_path_at_empty+0x49/0x70
 [<c118283f>] ? user_path_at+0x1f/0x30
 [<c11941e7>] path_getxattr+0x47/0x80
 [<c11948e7>] SyS_getxattr+0x27/0x30
 [<c163f748>] sysenter_do_call+0x12/0x12
Code: 66 90 55 89 e5 57 56 53 66 66 66 66 90 8b 78 20 89 d3 ba 67 b4 b5 f8 89 d8 89 ce e8 42 7c 7b c8 85 c0 75 16 0f b6 87 44 01 00
00 <88> 06 b8 01 00 00 00 5b 5e 5f 5d c3 8d 76 00 b8 ea ff ff ff eb
EIP: [<f8b54d69>] f2fs_xattr_advise_get+0x29/0x40 [f2fs] SS:ESP 0068:f19a7e08
CR2: 0000000000000000
---[ end trace 860260654f1f416a ]---

The reason is that in getfattr there are two steps which is indicated by strace info:
1) try to lookup and get size of specified xattr.
2) get value of the extented attribute.

strace info:
getxattr("/mnt/f2fs/file", "system.advise", 0x0, 0) = 1
getxattr("/mnt/f2fs/file", "system.advise", "\x00", 256) = 1

For the first step, getfattr may pass a NULL pointer in @value and zero in @SiZe
as parameters for ->getxattr, but we access this @value pointer directly without
checking whether the pointer is valid or not in f2fs_xattr_advise_get, so the
oops occurs.

This patch fixes this issue by verifying @value pointer before using.

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

This patch fixes to dirty inode for persisting i_advise of f2fs inode info into
on-disk inode if user sets system.advise through setxattr. Otherwise the new
value will be lost.

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

Map bh over max size which caller defined is not needed, limit it in
f2fs_map_bh.

Signed-off-by: Chao Yu <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

This patch fixes the below warning.

sparse warnings: (new ones prefixed by >>)

>> fs/f2fs/inode.c:56:23: sparse: restricted __le32 degrades to integer
>> fs/f2fs/inode.c:56:52: sparse: restricted __le32 degrades to integer

Reported-by: kbuild test robot <[email protected]>
Signed-off-by: Jaegeuk Kim <[email protected]>

Previously, inode page is not correctly locked and unlocked in pair during
the roll-forward recovery.

Signed-off-by: Jaegeuk Kim <[email protected]>

If f2fs was corrupted with missing dot dentries, it needs to recover them after
fsck.f2fs detection.

The underlying precedure is:

1. The fsck.f2fs remains F2FS_INLINE_DOTS flag in directory inode, if it detects
missing dot dentries.

2. When f2fs looks up the corrupted directory, it triggers f2fs_add_link with
proper inode numbers and their dot and dotdot names.

3. Once f2fs recovers the directory without errors, it removes F2FS_INLINE_DOTS
finally.

Signed-off-by: Jaegeuk Kim <[email protected]>

When assigning i_mode for dotdot, it needs to assign parent's i_mode.

Signed-off-by: Jaegeuk Kim <[email protected]>

If there are multiple fsynced dnodes having a dent flag, roll-forward routine
sets FI_INC_LINK for their inode, and recovery_dentry increases its link count
accordingly.
That results in normal file having a link count as 2, so we can't unlink those
files.

This was added to handle several inode blocks having same inode number with
different directory paths.
But, current f2fs doesn't replay all of path changes and only recover its dentry
for the last fsynced inode block.
So, there is no reason to do this.

Signed-off-by: Jaegeuk Kim <[email protected]>

During the roll-forward recovery, if we found a new data index written fsync
lastly, we need to recover new block address.
But, if that address was corrupted, we should not recover that.
Otherwise, f2fs gets kernel panic from:

 In check_index_in_prev_nodes(),

    sentry = get_seg_entry(sbi, segno);
             --------------------------> out-of-range segno.

Signed-off-by: Jaegeuk Kim <[email protected]>

Signed-off-by: Theodore Ts'o <[email protected]>

Signed-off-by: Michael Halcrow <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>

Required for future encryption xattr changes.

Signed-off-by: Michael Halcrow <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>

Signed-off-by: Michael Halcrow <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>

Signed-off-by: Michael Halcrow <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>
Signed-off-by: Ildar Muslukhov <[email protected]>

The page-types tool was relocated to tools/vm in the 3.4 kernel timeframe.

Signed-off-by: Randy Wright <[email protected]>
Signed-off-by: Jonathan Corbet <[email protected]>

Miss a ')' for ifneq in Makefile, the related building error:

  Documentation/blackfin/Makefile:2: *** invalid syntax in conditional.  Stop.
  make[1]: *** [Documentation/blackfin] Error 2

Signed-off-by: Chen Gang <[email protected]>
Signed-off-by: Jonathan Corbet <[email protected]>

Memcg/kmem reclaim support has been finally merged. Reflect this in the
documentation.

Acked-by: Michal Hocko <[email protected]>
Signed-off-by: Vladimir Davydov <[email protected]>
Signed-off-by: Jonathan Corbet <[email protected]>

On encrypt, we will re-assign the buffer_heads to point to a bounce
page rather than the control_page (which is the original page to write
that contains the plaintext). The block I/O occurs against the bounce
page.  On write completion, we re-assign the buffer_heads to the
original plaintext page.

On decrypt, we will attach a read completion callback to the bio
struct. This read completion will decrypt the read contents in-place
prior to setting the page up-to-date.

The current encryption mode, AES-256-XTS, lacks cryptographic
integrity. AES-256-GCM is in-plan, but we will need to devise a
mechanism for handling the integrity data.

Signed-off-by: Michael Halcrow <[email protected]>
Signed-off-by: Ildar Muslukhov <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>

Signed-off-by: Michael Halcrow <[email protected]>
Signed-off-by: Ildar Muslukhov <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>

Enforce the following inheritance policy:

1) An unencrypted directory may contain encrypted or unencrypted files
or directories.

2) All files or directories in a directory must be protected using the
same key as their containing directory.

As a result, assuming the following setup:

mke2fs -t ext4 -Fq -O encrypt /dev/vdc
mount -t ext4 /dev/vdc /vdc
mkdir /vdc/a /vdc/b /vdc/c
echo foo | e4crypt add_key /vdc/a
echo bar | e4crypt add_key /vdc/b
for i in a b c ; do cp /etc/motd /vdc/$i/motd-$i ; done

Then we will see the following results:

cd /vdc
mv a b			# will fail; /vdc/a and /vdc/b have different keys
mv b/motd-b a		# will fail, see above
ln a/motd-a b		# will fail, see above
mv c a	    		# will fail; all inodes in an encrypted directory
   	  		#	must be encrypted
ln c/motd-c b		# will fail, see above
mv a/motd-a c		# will succeed
mv c/motd-a a		# will succeed

Signed-off-by: Michael Halcrow <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>

Signed-off-by: Michael Halcrow <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>

Pulls block_write_begin() into fs/ext4/inode.c because it might need
to do a low-level read of the existing data, in which case we need to
decrypt it.

Signed-off-by: Michael Halcrow <[email protected]>
Signed-off-by: Ildar Muslukhov <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>

Signed-off-by: Michael Halcrow <[email protected]>
Signed-off-by: Ildar Muslukhov <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>

Signed-off-by: Uday Savagaonkar <[email protected]>
Signed-off-by: Ildar Muslukhov <[email protected]>
Signed-off-by: Michael Halcrow <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>

…ames

For encrypted directories, we need to pass in a separate parameter for
the decrypted filename, since the directory entry contains the
encrypted filename.

Signed-off-by: Theodore Ts'o <[email protected]>

Signed-off-by: Uday Savagaonkar <[email protected]>
Signed-off-by: Ildar Muslukhov <[email protected]>
Signed-off-by: Michael Halcrow <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>

Modifies dx_show_leaf and dx_probe to support fname encryption.
Filename encryption not yet enabled.

Signed-off-by: Uday Savagaonkar <[email protected]>
Signed-off-by: Ildar Muslukhov <[email protected]>
Signed-off-by: Michael Halcrow <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>

Modifies htree_dirblock_to_tree, dx_make_map, ext4_match search_dir,
and ext4_find_dest_de to support fname crypto.  Filename encryption
feature is not yet enabled at this patch.

Signed-off-by: Uday Savagaonkar <[email protected]>
Signed-off-by: Ildar Muslukhov <[email protected]>
Signed-off-by: Michael Halcrow <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>

Signed-off-by: Uday Savagaonkar <[email protected]>
Signed-off-by: Ildar Muslukhov <[email protected]>
Signed-off-by: Michael Halcrow <[email protected]>
Signed-off-by: Theodore Ts'o <[email protected]>

- don't lock lp->lock in the iss_net_timer for the call of iss_net_poll,
  it will lock it itself;
- invert order of lp->lock and opened_lock acquisition in the
  iss_net_open to make it consistent with iss_net_poll;
- replace spin_lock with spin_lock_bh when acquiring locks used in
  iss_net_timer from non-atomic context;
- replace spin_lock_irqsave with spin_lock_bh in the iss_net_start_xmit
  as the driver doesn't use lp->lock in the hard IRQ context;
- replace __SPIN_LOCK_UNLOCKED(lp.lock) with spin_lock_init, otherwise
  lockdep is unhappy about using non-static key.

Cc: <[email protected]>
Signed-off-by: Max Filippov <[email protected]>

Signed-off-by: Max Filippov <[email protected]>

Signed-off-by: Max Filippov <[email protected]>

The struct page is opaque for userspace tools, so it'd be better to save
pfn in order to identify page frames.

The textual output of $debugfs/tracing/trace file remains unchanged and
only raw (binary) data format is changed - but thanks to libtraceevent,
userspace tools which deal with the raw data (like perf and trace-cmd)
can parse the format easily.  So impact on the userspace will also be
minimal.

Signed-off-by: Namhyung Kim <[email protected]>
Based-on-patch-by: Joonsoo Kim <[email protected]>
Acked-by: Ingo Molnar <[email protected]>
Acked-by: Steven Rostedt <[email protected]>
Cc: David Ahern <[email protected]>
Cc: Jiri Olsa <[email protected]>
Cc: Minchan Kim <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: [email protected]
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>

The perf kmem command records and analyze kernel memory allocation only
for SLAB objects.  This patch implement a simple page allocator analyzer
using kmem:mm_page_alloc and kmem:mm_page_free events.

It adds two new options of --slab and --page.  The --slab option is for
analyzing SLAB allocator and that's what perf kmem currently does.

The new --page option enables page allocator events and analyze kernel
memory usage in page unit.  Currently, 'stat --alloc' subcommand is
implemented only.

If none of these --slab nor --page is specified, --slab is implied.

First run 'perf kmem record' to generate a suitable perf.data file:

  # perf kmem record --page sleep 5

Then run 'perf kmem stat' to postprocess the perf.data file:

  # perf kmem stat --page --alloc --line 10

  -------------------------------------------------------------------------------
   PFN              | Total alloc (KB) | Hits     | Order | Mig.type | GFP flags
  -------------------------------------------------------------------------------
            4045014 |               16 |        1 |     2 |  RECLAIM |  00285250
            4143980 |               16 |        1 |     2 |  RECLAIM |  00285250
            3938658 |               16 |        1 |     2 |  RECLAIM |  00285250
            4045400 |               16 |        1 |     2 |  RECLAIM |  00285250
            3568708 |               16 |        1 |     2 |  RECLAIM |  00285250
            3729824 |               16 |        1 |     2 |  RECLAIM |  00285250
            3657210 |               16 |        1 |     2 |  RECLAIM |  00285250
            4120750 |               16 |        1 |     2 |  RECLAIM |  00285250
            3678850 |               16 |        1 |     2 |  RECLAIM |  00285250
            3693874 |               16 |        1 |     2 |  RECLAIM |  00285250
   ...              | ...              | ...      | ...   | ...      | ...
  -------------------------------------------------------------------------------

  SUMMARY (page allocator)
  ========================
  Total allocation requests     :           44,260   [          177,256 KB ]
  Total free requests           :              117   [              468 KB ]

  Total alloc+freed requests    :               49   [              196 KB ]
  Total alloc-only requests     :           44,211   [          177,060 KB ]
  Total free-only requests      :               68   [              272 KB ]

  Total allocation failures     :                0   [                0 KB ]

  Order     Unmovable   Reclaimable       Movable      Reserved  CMA/Isolated
  -----  ------------  ------------  ------------  ------------  ------------
      0            32             .        44,210             .             .
      1             .             .             .             .             .
      2             .            18             .             .             .
      3             .             .             .             .             .
      4             .             .             .             .             .
      5             .             .             .             .             .
      6             .             .             .             .             .
      7             .             .             .             .             .
      8             .             .             .             .             .
      9             .             .             .             .             .
     10             .             .             .             .             .

Signed-off-by: Namhyung Kim <[email protected]>
Tested-by: Arnaldo Carvalho de Melo <[email protected]>
Cc: David Ahern <[email protected]>
Cc: Jiri Olsa <[email protected]>
Cc: Joonsoo Kim <[email protected]>
Cc: Minchan Kim <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: [email protected]
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>

Casual turbostat users generally just want to know MHz.
So by default, just print enough information to make sense of MHz.

All the other configuration data and columns for C-states and temperature etc,
are printed with the --debug option.

Signed-off-by: Len Brown <[email protected]>

cosmetic only.

order the decoding of MSR_PERF_LIMIT_REASONS bits
from MSB to LSB -- which you notice when more than 1 bit is set
and you are, say, comparing the output to the documentation...

Signed-off-by: Len Brown <[email protected]>

syntax only.

The cool kids are now using the phrase "base frequency",
where in the past we used "max non-turbo frequency" or "TSC frequency".

This distinction becomes important when a processor has a TSC
that runs at a different speed than the "base frequency".

Signed-off-by: Len Brown <[email protected]>

… mode

When perf probe searched in a debuginfo file and failed, it tried with
an alternative, in function get_alternative_probe_event():

        memcpy(tmp, &pev->point, sizeof(*tmp));
        memset(&pev->point, 0, sizeof(pev->point));

In this case, it drops the retprobe flag and forgets to set it back in
find_alternative_probe_point(), so the problem occurs.

Can be reproduced as following:

  $ perf probe -v -k vmlinux --add='sys_write%return'
  ...
  Added new event:
  Writing event: p:probe/sys_write _stext+1584952
    probe:sys_write      (on sys_write%return)

  $ cat /sys/kernel/debug/tracing/kprobe_events
  p:probe/sys_write _stext+1584952

After this patch:

  $ perf probe -v -k vmlinux --add='sys_write%return'
  Added new event:
  Writing event: r:probe/sys_write SyS_write+0
    probe:sys_write      (on sys_write%return)

  $ cat /sys/kernel/debug/tracing/kprobe_events
  r:probe/sys_write SyS_write

Signed-off-by: He Kuang <[email protected]>
Tested-by: Arnaldo Carvalho de Melo <[email protected]>
Acked-by: Masami Hiramatsu <[email protected]>
Cc: Namhyung Kim <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Wang Nan <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>

MSR_TURBO_RATIO_LIMIT has grown into a set of three registers.
Add the documented names for them, in preparation
for deleting the previous ad-hoc names:

+#define MSR_TURBO_RATIO_LIMIT          0x000001ad
+#define MSR_TURBO_RATIO_LIMIT1         0x000001ae
+#define MSR_TURBO_RATIO_LIMIT2         0x000001af

Signed-off-by: Len Brown <[email protected]>
Cc: [email protected]

If we use lazy matching, it failed to open a souce file if perf command
is invoked outside of compilation directory:

$ perf probe -a '__schedule;clear_*'
Failed to open kernel/sched/core.c: No such file or directory
  Error: Failed to add events. (-2)

OTOH, other commands like "probe -L" can solve the souce directory by
themselves. Let's make it possible for lazy matching too!

Signed-off-by: Naohiro Aota <[email protected]>
Acked-by: Masami Hiramatsu <[email protected]>
Cc: He Kuang <[email protected]>
Cc: Jiri Olsa <[email protected]>
Cc: Namhyung Kim <[email protected]>
Cc: Paul Mackerras <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>

The first argument passed to find_probe_point_lazy() should be CU die,
which will be passed to die_walk_lines() when lazy_line matches.
Currently, when we probe with lazy_line pattern to file without function
name, NULL pointer is passed and causes a segment fault.

Can be reproduced as following:

  $ perf probe -k vmlinux --add='fs/super.c;s->s_count=1;'
  [ 1958.984658] perf[1020]: segfault at 10 ip 00007fc6e10d8c71 sp
  00007ffcbfaaf900 error 4 in libdw-0.161.so[7fc6e10ce000+34000]
  Segmentation fault

After this patch:

  $ perf probe -k vmlinux --add='fs/super.c;s->s_count=1;'
  Added new event:
  probe:_stext         (on @fs/super.c)

  You can now use it in all perf tools, such as:
    perf record -e probe:_stext -aR sleep 1

Signed-off-by: He Kuang <[email protected]>
Tested-by: Arnaldo Carvalho de Melo <[email protected]>
Acked-by: Masami Hiramatsu <[email protected]>
Cc: Namhyung Kim <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Wang Nan <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>

Linux 4.0

…ux-xtensa into for_next

Xtensa improvements for 4.1:

- fix locking issues in ISS network driver;
- document PIC and MX interrupt distributor device tree bindings;
- add CY7C67300 USB controller support to XTFPGA.

Signed-off-by: Chris Zankel <[email protected]>

…ux/kernel/git/acme/linux into perf/urgent

Pull perf/core improvements and fixes from Arnaldo Carvalho de Melo:

New features:

  - Analyze page allocator events in 'perf kmem' (Namhyung Kim)

User visible changes:

  - Fix retprobe 'perf probe' handling when failing to find needed debuginfo (He Kuang)

  - lazy_line probe fixes in 'perf probe' (Naohiro Aota, He Kuang)

Infrastructure changes:

  - Record pfn instead of pointer to struct page in tracepoints (Namhyung Kim)

Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>

This patch reduces source code differences between the Linux kernel and the
ACPICA upstream so that the linuxized ACPICA 20150410 release can be
applied with reduced human intervention.

Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

…address.

ACPICA commit 7d9fd64397d7c38899d3dc497525f6e6b044e0e3

OSPMs like Linux expect an acpi_physical_address returning value from
acpi_find_root_pointer(). This triggers warnings if sizeof (acpi_size) doesn't
equal to sizeof (acpi_physical_address):
  drivers/acpi/osl.c:275:3: warning: passing argument 1 of 'acpi_find_root_pointer' from incompatible pointer type [enabled by default]
  In file included from include/acpi/acpi.h:64:0,
                   from include/linux/acpi.h:36,
                   from drivers/acpi/osl.c:41:
  include/acpi/acpixf.h:433:1: note: expected 'acpi_size *' but argument is of type 'acpi_physical_address *'
This patch corrects acpi_find_root_pointer().

Link: acpica/acpica@7d9fd643
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

ACPICA commit 9e2d8180f4d5e61949b17513bae8aff6412f62dd

The offset calculation needn't convert a pointer to a special integer type.
So this patch uses ACPI_TO_INTEGER() instead.

This patch only affects acpidump tool.

Link: acpica/acpica@9e2d8180
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

ACPICA commit c327986430b7eb170c17d0f45735fe71bb62a810

The conversion converts an acpi_physical_address to acpi_physical_address,
this patch thus removes such useless conversion.

Link: acpica/acpica@c3279864
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

…_TO_PHYSADDR().

ACPICA commit 154f6d074dd38d6ebc0467ad454454e6c5c9ecdf

There are code pieces converting pointers using "(acpi_physical_address) x"
or "ACPI_CAST_PTR (t, x)" formats, this patch cleans up them.

Known issues:
1. Cleanup of "(ACPI_PHYSICAL_ADDRRESS) x" for a table field
   For the conversions around the table fields, it is better to fix it with
   alignment also fixed. So this patch doesn't modify such code. There
   should be no functional problem by leaving them unchanged.

Link: acpica/acpica@154f6d07
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

ACPICA commit 7f06739db43a85083a70371c14141008f20b2198

For physical addresses, since the address may exceed 32-bit address range
after calculation, we should use %8.8X%8.8X (see ACPI_FORMAT_UINT64()) to
convert the %p formats.

This is a preparation to switch acpi_physical_address to 64-bit on 32-bit
kernel builds.

Link: acpica/acpica@7f06739d
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

…elpers.

ACPICA commit b60612373a4ef63b64a57c124576d7ddb6d8efb6

For physical addresses, since the address may exceed 32-bit address range
after calculation, we should use 0x%8.8X%8.8X instead of ACPI_PRINTF_UINT
and ACPI_FORMAT_UINT64() instead of
ACPI_FORMAT_NATIVE_UINT()/ACPI_FORMAT_TO_UINT().

This patch also removes above replaced macros as there are no users.

This is a preparation to switch acpi_physical_address to 64-bit on 32-bit
kernel builds.

Link: acpica/acpica@b6061237
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

ACPICA commit aacf863cfffd46338e268b7415f7435cae93b451

It is reported that on a physically 64-bit addressed machine, 32-bit kernel
can trigger crashes in accessing the memory regions that are beyond the
32-bit boundary. The region field's start address should still be 32-bit
compliant, but after a calculation (adding some offsets), it may exceed the
32-bit boundary. This case is rare and buggy, but there are real BIOSes
leaked with such issues (see References below).

This patch fixes this gap by always defining IO addresses as 64-bit, and
allows OSPMs to optimize it for a real 32-bit machine to reduce the size of
the internal objects.

Internal acpi_physical_address usages in the structures that can be fixed
by this change include:
 1. struct acpi_object_region:
    acpi_physical_address		address;
 2. struct acpi_address_range:
    acpi_physical_address		start_address;
    acpi_physical_address		end_address;
 3. struct acpi_mem_space_context;
    acpi_physical_address		address;
 4. struct acpi_table_desc
    acpi_physical_address		address;
See known issues 1 for other usages.

Note that acpi_io_address which is used for ACPI_PROCESSOR may also suffer
from same problem, so this patch changes it accordingly.

For iasl, it will enforce acpi_physical_address as 32-bit to generate
32-bit OSPM compatible tables on 32-bit platforms, we need to define
ACPI_32BIT_PHYSICAL_ADDRESS for it in acenv.h.

Known issues:
 1. Cleanup of mapped virtual address
   In struct acpi_mem_space_context, acpi_physical_address is used as a virtual
   address:
    acpi_physical_address                   mapped_physical_address;
   It is better to introduce acpi_virtual_address or use acpi_size instead.
   This patch doesn't make such a change. Because this should be done along
   with a change to acpi_os_map_memory()/acpi_os_unmap_memory().
   There should be no functional problem to leave this unchanged except
   that only this structure is enlarged unexpectedly.

Link: acpica/acpica@aacf863c
Reference: https://bugzilla.kernel.org/show_bug.cgi?id=87971
Reference: https://bugzilla.kernel.org/show_bug.cgi?id=79501
Reported-and-tested-by: Paul Menzel <[email protected]>
Reported-and-tested-by: Sial Nije <[email protected]>
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Bob Moore <[email protected]>
Cc: All applicable <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

…dard_table().

ACPICA commit c70434d4da13e65b6163c79a5aa16b40193631c7

ACPI_MTX_TABLES is acquired and released by the callers of
acpi_tb_install_standard_table() so releasing it in the function itself is
causing the following error in Linux kernel if the table is reloaded:

ACPI Error: Mutex [0x2] is not acquired, cannot release (20141107/utmutex-321)
Call Trace:
  [<ffffffff81b0bd48>] dump_stack+0x4f/0x7b
  [<ffffffff81546bf5>] acpi_ut_release_mutex+0x47/0x67
  [<ffffffff81544357>] acpi_load_table+0x73/0xcb

Link: acpica/acpica@c70434d4
Signed-off-by: Octavian Purdila <[email protected]>
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Bob Moore <[email protected]>
Cc: 3.16+ <[email protected]> # 3.16+
Signed-off-by: Rafael J. Wysocki <[email protected]>

…ues for GPE and fixed event.

ACPICA commit e25d791e4b3d5b9f4ead298269610cb05f89749a

There is a facility in Linux, developers can obtain GPE and fixed event
status via /sys/firmware/interrupts/. This is implemented using
acpi_get_event_status() and acpi_get_gpe_status(). Recently while debugging some
GPE race issues, it is found that the facility is lacking in the ability to
obtain real hardware register values, the confusing information makes
debugging difficult.

This patch modifies acpi_get_gpe_status() to return EN register values to fix
this gap. Then flags returned from acpi_get_event_status() and
acpi_get_gpe_status() are also cleaned up to reflect this change.

The old ACPI_EVENT_FLAG_SET is carefully kept to avoid regressions. It can
be deleted after we can make sure all its references are removed from OSPM
code. Lv Zheng.

Link: acpica/acpica@e25d791e
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

ACPICA commit 3d9fb6d1f216a78ad098d3ad23f1304376c2f4ef

The macro __DATE__ and friends is not allowed in the Linux kernel. Also,
including the build time in output doesn't seem to provide any value.

Link: acpica/acpica@3d9fb6d1
Signed-off-by: Rasmus Villemoes <[email protected]>
Acked-by: David E. Box <[email protected]>
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

ACPICA commit 2a9ebd974aee41391f4b0edcd4f0cc5ee23ec2f8

Remove the function acpi_ut_create_pkg_state_and_push() that is not used anywhere.
This was partially found by using a static code analysis program called cppcheck.

Link: acpica/acpica@2a9ebd97
Signed-off-by: Rickard Strandqvist <[email protected]>
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

…ile.

ACPICA commit 6eb364d790dd103bd4990f808e0095a421c437cb

acpi_tb_store_table() implements a logic that is only correct to iasl. So it
won't be used by any other utilities except iasl. This function is
complained by the kernel users as an unused function. The best choice to
stop releasing it to the Linux kernel should be moving it to adisasm.c.

ACPI table manager can use both struct acpi_table_desc (direct referencing)
and table index (indirect referencing) as the descriptor to the table, so
acpi_tb_get_next_root_index() is extended to return both of them to allow
maximum usability from the callers. NOTE that indirect referencing is a
design result to meet the boot stage static allocation requirement for the
table descriptors.

This is a linuxized acpi_tb_store_table() removing result, there should be
no functional changes introduced to the Linux kernel by this patch except
the additonal kernel unused argument for acpi_tb_get_next_root_index()
(renamed to acpi_tb_get_next_root_index()). This argument is used in the
ACPICA upstream.

Link: acpica/acpica@6eb364d7
Reported-by: Rickard Strandqvist <[email protected]>
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

ACPICA commit 9a5982afbebc56289c4834b5f6dac87e0f04af14

Some conditional compilation definitions are wrong across header and source
files. This patch corrects them for the utilities component.

Link: acpica/acpica@9a5982af
Reported-by: Rickard Strandqvist <[email protected]>
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

ACPICA commit f92a08512b0b35dce4a7fc6a73216674a3c2541b

Some conditional compilation definitions are wrong across header and source
files. This patch corrects them for the resources component.

NOTE that a further patch in this patchset cleans up all ACPI_EXEC_APP,
converting them into ACPI_DEBUGGER, including part of this change.

Link: acpica/acpica@f92a0851
Reported-by: Rickard Strandqvist <[email protected]>
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

ACPICA commit 46dc081e570b1363af1e368980201cbb65c4100f

Update for some recent changes, detected by MSVC and FreeBSD builds.

Link: acpica/acpica@46dc081e
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

ACPICA commit 84f3569db7accc576ace2dae81d101467254fe9d

Was using %d instead of properly using %u.

This patch only affects acpidump tool.

Link: acpica/acpica@84f3569d
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

ACPICA commit 184f3cc4d162a6b6b2005eacd8be2fe55f19a245

- Change global #ifdef to check for ACPI_DEBUGGER only.
- Cleanup some long lines and misaligned code.

Link: acpica/acpica@184f3cc4
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

ACPICA commit f65d0e5b880860a6da0c336b7c48139d1469f5b4

- Remove unused variables
- Update to use standard naming/capitalization convention.

Linux kernel is not affected by this patch.

Link: acpica/acpica@f65d0e5b
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

…AML.

ACPICA commit f5d2ff3fa05341d9fe27250fcb2e14c7f871432c

For dynamically loaded tables that have unknown ACPI signatures,
this option (-df) forces the disassembler to treat the table as if
it contains valid AML code (like a DSDT or SSDT). Otherwise, the
disassembler will treat the table as an unkown data table and
simply do a hex dump of the table.

Linux kernel is not affected by this patch.

Link: acpica/acpica@f5d2ff3f
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

ACPICA commit 4f9e950d41cd6f6c704d5d6cf518647620d65e99

Add support to fold expressions with a target operand -- these are
folded and converted to a Store operator:

Add (4, 3, INT1) --> Store (7, INT1)

Also supports ASL+ constructs:

INT1 = 4 + 3 --> Store (7, INT1)

Linux kernel is not affected by this patch.

Link: acpica/acpica@4f9e950d
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

ACPICA commit d115fe2ffdab449d6107d58580c5afd0a81d65fe

This change adds the basic low-level infrastructure for the External
AML opcode. The interpreter will simply ignore this op, as the op
is intended for use by the disassembler only.

Note that External() opcode is useful for disassembler, interpreter
can simply ignore it and still return exceptions for unknown control
methods so the kernel part only includes the grammar definition of
External() opcode in order to ignore it but doesn't interpret it.

Link: acpica/acpica@d115fe2f
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>

ACPICA commit b293f602a67da478ae0bec129e68bd99787d9908

This change adds this string for Windows 10.

Link: acpica/acpica@b293f602
Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Lv Zheng <[email protected]>
Signed-off-by: Rafael J. Wysocki <[email protected]>