drm/amd/powerplay: Remove useless variable

The result variable is initialized at the beginning of the function, but
its value does not change during the function execution. Thus, remove the
variable and return the SUCCESS value, which is 0.

Issue found by coccinelle script:
* Unneeded variable: "result". Return "0"

Path to the cocci script: scripts/coccinelle/misc/returnvar.cocci

Signed-off-by: Georgiana Chelu <georgiana.chelu93@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: Don't cast kzalloc() return value

The kzalloc function returns a void pointer and the assignment
operator converts it to the type of pointer it is assigned to.
Therefore, there is no need to cast.

Issue found by alloc_cast.cocci:
* WARNING: casting value returned by memory allocation function
to <struct type> is useless.

Path to the cocci script: scripts/coccinelle/api/alloc/alloc_cast.cocci

Signed-off-by: Georgiana Chelu <georgiana.chelu93@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: allow GTT overcommit during bind

While binding BOs to GART we need to allow a bit overcommit in the GTT
domain. Otherwise we can never use the full GART space when GART size=GTT size.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: linear validate first then bind to GART

For VM emulation for old UVD/VCE we need to validate the BO with linear
VRAM flag set first and then eventually bind it to GART.

Validating with linear VRAM flag set can move the BO to GART making
UVD/VCE read/write from an unbound GART BO.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
CC: stable@vger.kernel.org
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/pp: Fix overflow when setup decf/pix/disp dpm table.

Clear the count in the single table setup function to
avoid missing any tables.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/pp: thermal control not enabled on vega10.

regression issue.
caused by "f12f9f5e5d455edebc01"
forget to set start_thermal_controller function point.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: busywait KIQ register accessing (v4)

Register accessing is performed when IRQ is disabled. Never sleep in
this function.

Known issue: dead sleep in many use cases of index/data registers.

v2:
 - wrap polling fence functions.
 - don't trigger IRQ for polling in case of wrongly fence signal.

v3:
 - handle wrap round gracefully.
 - add comments for polling function

v4:
 - don't return negative timeout confused with error code

Signed-off-by: pding <Pixel.Ding@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: report more amdgpu_fence_info

Only for GFX ring. This can help checking MCBP feature.

The fence at the end of the frame will indicate the completion status.
If the frame completed normally, the fence is written to the address
given in the EVENT_WRITE_EOP packet. If preemption occurred in the
previous IB the address is adjusted by 2 DWs. If work submitted in the
frame was reset before completion, the fence address is adjusted by
four DWs. In the case that preemption occurred, and before preemption
completed a reset was initiated, the address will be adjusted with six
DWs

Signed-off-by: pding <Pixel.Ding@amd.com>
Reviewed-by: Monk Liu <monk.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:don't check soft_reset for sriov

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Ack-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:fix duplicated setting job's vram_lost

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:reduce wb to 512 slot

with current WB usage we only use 57 slots, so 512
is extreamly sufficient, and reduce to 512 can
make WB fit into one page.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fix regresstion on SR-IOV gpu reset failed

fw ucode is corrupted after vf flr by PSP so ucode_init() is
a must in psp_hw_init othewise KIQ/KCQ enabling will fail

Revert "drm/amdgpu: refine code delete duplicated error handling"
This reverts commit e57b87ff828f95efe992468e6d18c2c059b27aa9.
Revert "drm/amdgpu: move amdgpu_ucode_init_bo to amdgpu_device.c"
This reverts commit 815b8f8595148d06a64d2ce4282e8e80dfcb02f1.

Reviewed-by: Monk Liu <monk.liu@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: Tidy up cz_dpm_powerup_vce()

Use PP_CAP macro.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: Tidy up cz_dpm_powerdown_vce()

Use PP_CAP macro.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: Tidy up cz_dpm_update_vce_dpm()

Use PP_CAP and tidy up indentation.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: Tidy up cz_dpm_update_uvd_dpm()

Use PP_CAP and tidy up indentation.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: Tidy up cz_dpm_powerup_uvd()

Use PP_CAP and simplify enable/disable logic.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: Tidy up cz_dpm_powerdown_uvd()

Use PP_CAP and tidy up indentation.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: Tidy up cz_start_dpm()

Remove unused variables.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: use ARRAY_SIZE

Using the ARRAY_SIZE macro improves the readability of the code.

Found with Coccinelle with the following semantic patch:
@r depends on (org || report)@
type T;
T[] E;
position p;
@@
(
 (sizeof(E)@p /sizeof(*E))
|
 (sizeof(E)@p /sizeof(E[...]))
|
 (sizeof(E)@p /sizeof(T))
)

Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Thierry Reding <treding@nvidia.com>
Signed-off-by: Jérémy Lefaure <jeremy.lefaure@lse.epita.fr>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: reserve root PD while releasing it

Otherwise somebody could try to evict it at the same time and try to use
half torn down structures.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-and-Tested-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: fix the fix for huge compound pages

We don't use compound pages at the moment. Take this into account when
freeing them.

Signed-off-by: Christian König <christian.koenig@amd.comd>
Reviewed-and-Tested-by: Michel Dänzer <michel.daenzer@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: bump version for vram lost counter query (v2)

v2: vram -> VRAM in comment

Reviewed-by: Michel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: minor CS optimization

We only need to loop over all IBs for old UVD/VCE command stream patching.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/sched: fix job tear down order v2

Move the trace before we signal the scheduler fence and drop the
scheduler fence reference directly before we free the job.

v2: keep extra s_fence reference

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Liu, Monk <Monk.Liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Add amdgpu_find_mm_node()

Replace some commonly repeated code with a function.

v2: Use amdgpu_find_mm_node() in amdgpu_ttm_io_mem_pfn()

Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Refactor amdgpu_move_blit

Add more generic function amdgpu_copy_ttm_mem_to_mem() that supports
arbitrary copy size, offsets and two BOs (source & dest.).

This is useful for KFD Cross Memory Attach feature where data needs to
be copied from BOs from different processes

v2: Add struct amdgpu_copy_mem and changed amdgpu_copy_ttm_mem_to_mem()
function parameters to use the struct

v3: Minor function name change

Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Fix extra call to amdgpu_ctx_put.

In amdgpu_cs_parser_init() in case of error handling
amdgpu_ctx_put() is called without setting p->ctx to NULL after that,
later amdgpu_cs_parser_fini() also calls amdgpu_ctx_put() again and
mess up the reference count.

Signed-off-by: Andrey Grodzovsky <Andrey.Grodzovsky@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: Fix unused variables with huge page support

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add VRAM lost query

Allows userspace to figure out if VRAM was lost.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: set -ECANCELED when dropping jobs

And return from the wait functions the fence error code.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: move the VRAM lost counter per context

Instead of per device track the VRAM lost per context and return ECANCELED
instead of ENODEV.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: keep copy of VRAM lost counter in job

Instead of reading the current counter from fpriv.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: partial revert VRAM lost handling v2

Keep blocking the CS, but revert everything else. Mapping BOs and info IOCTL
are harmless and can still happen even when VRAM content ist lost.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: add transparent huge page support for wc or uc allocations v2

Add a new huge page pool and try to allocate from it when it makes sense.

v2: avoid compound pages for now

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: move more logic into ttm_page_pool_get_pages

Make it easier to add huge page pool.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: add transparent huge page support for cached allocations v2

Try to allocate huge pages when it makes sense.

v2: avoid compound pages for now

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: don't use compound pages for now

We need to figure out first how to correctly map them into the CPU page tables.

bug: https://bugs.freedesktop.org/show_bug.cgi?id=103138
Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: correct reference clock value on vega10

Old value from bringup was wrong.

Cc: stable@vger.kernel.org
Signed-off-by: Ken Wang <Ken.Wang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: SR-IOV data exchange between PF&VF

SR-IOV need to exchange some data between PF&VF through shared VRAM

PF will copy some necessary firmware and information to the shared
VRAM. It also requires some information from VF. PF will send a
key through mailbox2 to help guest calculate checksum so that it can
verify whether the data is correct.

So check the data on the specified offset of the shared VRAM, if the
checksum is right, read values from it and write some VF information
next to the data from PF.

Signed-off-by: Horace Chen <horace.chen@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Move old fence waiting before reservation lock is aquired v2

Helps avoiding deadlock during GPU reset.
Added mutex to amdgpu_ctx to preserve order of fences on a ring.

v2:
Put waiting logic in a function in a seperate function in amdgpu_ctx.c

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Refactor amdgpu_cs_ib_vm_chunk and amdgpu_cs_ib_fill.

This enables old fence waiting before reservation lock is aquired
which in turn is part of a bigger solution to deadlock happening
when gpu reset with VRAM recovery accures during intensive rendering.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: make function uvd_v6_0_enc_get_destroy_msg static

The function uvd_v6_0_enc_get_destroy_msg is local to the source and
does not need to be in global scope, so make it static.

Cleans up sparse warning:
symbol 'uvd_v6_0_enc_get_destroy_msg' was not declared. Should it be
static?

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Colin Ian King <colin.king@canonical.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/pp: remove polaris10_smc/smumgr split.

move functions in polaris_smc.c to
polaris10_smumgr.c and make all functions in
polaris10_smumgr.c static.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/pp: remove tonga_smc/smumgr split.

move functions in tonga_smc.c to tonga_smumgr.c
and make all functions in tonga_smumgr.c static.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/pp: remove iceland_smc/smumgr split.

move functions in iceland_smc.c to iceland_smumgr.c
and make all functions in iceland_smumgr.c static.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/pp: remove fiji_smc/smumgr split.

make all functions in fiji_smumgr.c static and
exported by pp_smumgr_func table.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/pp: rename ci_smc.c to ci_smumgr.c

for consistency.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: refine code delete duplicated error handling

in function amdgpu_ucode_init_bo, when failed, it will
set load_type to AMDGPU_FW_LOAD_DIRECT.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/pp: move common function to smu7_smumgr.c

fiji and polaris can share same setup_pwr_virus
function.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/pp: implement function notify_cac_buffer_info on VI

Used for smu power logging.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/pp: implement function notify_cac_buffer_info on Vega

Used for smu power logging.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/pp: add new function point in hwmgr.

used for notify SMU the allocated buffer address.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/pp: export new smu messages for vega10

New messages for smu power logging.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: get the temperature on CZ

Setting the function pointer to the get the temperature on CZ.

Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Satyajit Sahu <satyajit.sahu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: bump version for new AMDGPU_SCHED ioctl

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: rename context priority levels

Don't leak implementation details about how each priority behaves to
usermode. This allows greater flexibility in the future.

Squash into c2636dc53abd8269a0930bccd564f2f195dba729

Signed-off-by: Andres Rodriguez <andresx7@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add interface for editing a foreign process's priority v3

The AMDGPU_SCHED_OP_PROCESS_PRIORITY_OVERRIDE ioctls are used to set
the priority of a different process in the current system.

When a request is dropped, the process's contexts will be
restored to the priority specified at context creation time.

A request can be dropped by setting the override priority to
AMDGPU_CTX_PRIORITY_UNSET.

An fd is used to identify the remote process. This is simpler than
passing a pid number, which is vulnerable to re-use, etc.

This functionality is limited to DRM_MASTER since abuse of this
interface can have a negative impact on the system's performance.

v2: removed unused output structure
v3: change refcounted interface for a regular set operation

Signed-off-by: Andres Rodriguez <andresx7@gmail.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add plumbing for ctx priority changes v2

Introduce amdgpu_ctx_priority_override(). A mechanism to override a
context's priority.

An override can be terminated by setting the override to
AMD_SCHED_PRIORITY_UNSET.

v2: change refcounted interface for a direct set

Signed-off-by: Andres Rodriguez <andresx7@gmail.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: introduce AMDGPU_CTX_PRIORITY_UNSET

Use _INVALID to identify bad parameters and _UNSET to represent the
lack of interest in a specific value.

Signed-off-by: Andres Rodriguez <andresx7@gmail.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/sched: allow clients to edit an entity's rq v2

This is useful for changing an entity's priority at runtime.

v2: don't modify the order of amd_sched_entity members

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Andres Rodriguez <andresx7@gmail.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: make amdgpu_to_sched_priority detect invalid parameters

Returning invalid priorities as _NORMAL is a backwards compatibility
quirk of amdgpu_ctx_ioctl(). Move this detail one layer up where it
belongs.

Signed-off-by: Andres Rodriguez <andresx7@gmail.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: implement ring set_priority for gfx_v8 compute v9

Programming CP_HQD_QUEUE_PRIORITY enables a queue to take priority over
other queues on the same pipe. Multiple queues on a pipe are timesliced
so this gives us full precedence over other queues.

Programming CP_HQD_PIPE_PRIORITY changes the SPI_ARB_PRIORITY of the
wave as follows:
        0x2: CS_H
        0x1: CS_M
        0x0: CS_L

The SPI block will then dispatch work according to the policy set by
SPI_ARB_PRIORITY. In the current policy CS_H is higher priority than
gfx.

In order to prevent getting stuck in loops of resources bouncing between
GFX and high priority compute and introducing further latency, we
statically reserve a portion of the pipe.

v2: fix srbm_select to ring->queue and use ring->funcs->type
v3: use AMD_SCHED_PRIORITY_* instead of AMDGPU_CTX_PRIORITY_*
v4: switch int to enum amd_sched_priority
v5: corresponding changes for srbm_lock
v6: change CU reservation to PIPE_PERCENT allocation
v7: use kiq instead of MMIO
v8: back to MMIO, and make the implementation sleep safe.
v9: corresponding changes for splitting HIGH into _HW/_SW

Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Andres Rodriguez <andresx7@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add framework for HW specific priority settings v9

Add an initial framework for changing the HW priorities of rings. The
framework allows requesting priority changes for the lifetime of an
amdgpu_job. After the job completes the priority will decay to the next
lowest priority for which a request is still valid.

A new ring function set_priority() can now be populated to take care of
the HW specific programming sequence for priority changes.

v2: set priority before emitting IB, and take a ref on amdgpu_job
v3: use AMD_SCHED_PRIORITY_* instead of AMDGPU_CTX_PRIORITY_*
v4: plug amdgpu_ring_restore_priority_cb into amdgpu_job_free_cb
v5: use atomic for tracking job priorities instead of last_job
v6: rename amdgpu_ring_priority_[get/put]() and align parameters
v7: replace spinlocks with mutexes for KIQ compatibility
v8: raise ring priority during cs_ioctl, instead of job_run
v9: priority_get() before push_job()

Reviewed-by: Christian König <christian.koenig@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Andres Rodriguez <andresx7@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add parameter to allocate high priority contexts v11

Add a new context creation parameter to express a global context priority.

The priority ranking in descending order is as follows:
 * AMDGPU_CTX_PRIORITY_HIGH_HW
 * AMDGPU_CTX_PRIORITY_HIGH_SW
 * AMDGPU_CTX_PRIORITY_NORMAL
 * AMDGPU_CTX_PRIORITY_LOW_SW
 * AMDGPU_CTX_PRIORITY_LOW_HW

The driver will attempt to schedule work to the hardware according to
the priorities. No latency or throughput guarantees are provided by
this patch.

This interface intends to service the EGL_IMG_context_priority
extension, and vulkan equivalents.

Setting a priority above NORMAL requires CAP_SYS_NICE or DRM_MASTER.

v2: Instead of using flags, repurpose __pad
v3: Swap enum values of _NORMAL _HIGH for backwards compatibility
v4: Validate usermode priority and store it
v5: Move priority validation into amdgpu_ctx_ioctl(), headline reword
v6: add UAPI note regarding priorities requiring CAP_SYS_ADMIN
v7: remove ctx->priority
v8: added AMDGPU_CTX_PRIORITY_LOW, s/CAP_SYS_ADMIN/CAP_SYS_NICE
v9: change the priority parameter to __s32
v10: split priorities into _SW and _HW
v11: Allow DRM_MASTER without CAP_SYS_NICE

Reviewed-by: Emil Velikov <emil.l.velikov@gmail.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Andres Rodriguez <andresx7@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: introduce AMDGPU_GEM_CREATE_EXPLICIT_SYNC v2

Introduce a flag to signal that access to a BO will be synchronized
through an external mechanism.

Currently all buffers shared between contexts are subject to implicit
synchronization. However, this is only required for protocols that
currently don't support an explicit synchronization mechanism (DRI2/3).

This patch introduces the AMDGPU_GEM_CREATE_EXPLICIT_SYNC, so that
users can specify when it is safe to disable implicit sync.

v2: only disable explicit sync in amdgpu_cs_ioctl

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Andres Rodriguez <andresx7@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add helper to convert a ttm bo to amdgpu_bo

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Andres Rodriguez <andresx7@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add VM support for huge pages v2

Convert GTT mappings into linear ones for huge page handling.

v2: use fragment size as minimum for linear conversion

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: DMA map/unmap consecutive pages as a whole v2

Instead of mapping them bit by bit map/unmap all consecutive
pages as in one call.

v2: test for consecutive pages instead of using compound page order.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: allocate/free multiple pages in a single call

Totally surprisingly this is more efficient than doing it page by page.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Reserve shared memory on VRAM for SR-IOV

SR-IOV need to reserve a piece of shared VRAM at the exact place
to exchange data betweem PF and VF. The start address and size of
the shared mem are passed to guest through VBIOS structure
VRAM_UsageByFirmware.

VRAM_UsageByFirmware is a general feature in VBIOS, it indicates
that VBIOS need to reserve a piece of memory on the VRAM.

Because the mem address is specified. Reserve it early in
amdgpu_ttm_init to make sure that it can monoplize the space.

Signed-off-by: Horace Chen <horace.chen@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Set the correct value for PDEs/PTEs of ATC memory on Raven

Without the additional bits set in PDEs/PTEs, the ATC memory access
would have failed on Raven.

Signed-off-by: Yong Zhao <yong.zhao@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

Merge branch 'drm-next-4.15' of git://people.freedesktop.org/~agd5f/linux into drm-next

More new stuff for 4.15. Highlights:
- Add clock query interface for raven
- Add new FENCE_TO_HANDLE ioctl
- UVD video encode ring support on polaris
- transparent huge page DMA support
- deadlock fixes
- compute pipe lru tweaks
- powerplay cleanups and regression fixes
- fix duplicate symbol issue with radeon and amdgpu
- misc bug fixes

* 'drm-next-4.15' of git://people.freedesktop.org/~agd5f/linux: (72 commits)
  drm/radeon/dp: make radeon_dp_get_dp_link_config static
  drm/radeon: move ci_send_msg_to_smc to where it's used
  drm/amd/sched: fix deadlock caused by unsignaled fences of deleted jobs
  drm/amd/sched: NULL out the s_fence field after run_job
  drm/amd/sched: move adding finish callback to amd_sched_job_begin
  drm/amd/sched: fix an outdated comment
  drm/amd/sched: rename amd_sched_entity_pop_job
  drm/amdgpu: minor coding style fix
  drm/ttm: add transparent huge page support for DMA allocations v2
  drm/ttm: add support for different pool sizes
  drm/ttm: remove unsued options from ttm_mem_global_alloc_page
  drm/amdgpu: add uvd enc irq
  drm/amdgpu: add uvd enc ib test
  drm/amdgpu: add uvd enc ring test
  drm/amdgpu: add uvd enc vm functions (v2)
  drm/amdgpu: add uvd enc into run queue
  drm/amdgpu: add uvd enc rings
  drm/amdgpu: add new uvd enc ring methods
  drm/amdgpu: add uvd enc command in header
  drm/amdgpu: add uvd enc registers in header
  ...

drm/radeon/dp: make radeon_dp_get_dp_link_config static

It's not used outside this file any longer.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/radeon: move ci_send_msg_to_smc to where it's used

It's used in ci_dpm.c so move it there and make it static.

Reviewed-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/sched: fix deadlock caused by unsignaled fences of deleted jobs

Highly concurrent Piglit runs can trigger a race condition where a pending
SDMA job on a buffer object is never executed because the corresponding
process is killed (perhaps due to a crash). Since the job's fences were
never signaled, the buffer object was effectively leaked. Worse, the
buffer was stuck wherever it happened to be at the time, possibly in VRAM.

The symptom was user space processes stuck in interruptible waits with
kernel stacks like:

    [<ffffffffbc5e6722>] dma_fence_default_wait+0x112/0x250
    [<ffffffffbc5e6399>] dma_fence_wait_timeout+0x39/0xf0
    [<ffffffffbc5e82d2>] reservation_object_wait_timeout_rcu+0x1c2/0x300
    [<ffffffffc03ce56f>] ttm_bo_cleanup_refs_and_unlock+0xff/0x1a0 [ttm]
    [<ffffffffc03cf1ea>] ttm_mem_evict_first+0xba/0x1a0 [ttm]
    [<ffffffffc03cf611>] ttm_bo_mem_space+0x341/0x4c0 [ttm]
    [<ffffffffc03cfc54>] ttm_bo_validate+0xd4/0x150 [ttm]
    [<ffffffffc03cffbd>] ttm_bo_init_reserved+0x2ed/0x420 [ttm]
    [<ffffffffc042f523>] amdgpu_bo_create_restricted+0x1f3/0x470 [amdgpu]
    [<ffffffffc042f9fa>] amdgpu_bo_create+0xda/0x220 [amdgpu]
    [<ffffffffc04349ea>] amdgpu_gem_object_create+0xaa/0x140 [amdgpu]
    [<ffffffffc0434f97>] amdgpu_gem_create_ioctl+0x97/0x120 [amdgpu]
    [<ffffffffc037ddba>] drm_ioctl+0x1fa/0x480 [drm]
    [<ffffffffc041904f>] amdgpu_drm_ioctl+0x4f/0x90 [amdgpu]
    [<ffffffffbc23db33>] do_vfs_ioctl+0xa3/0x5f0
    [<ffffffffbc23e0f9>] SyS_ioctl+0x79/0x90
    [<ffffffffbc864ffb>] entry_SYSCALL_64_fastpath+0x1e/0xad
    [<ffffffffffffffff>] 0xffffffffffffffff

Note: The correctness of this change depends on the earlier commit
"drm/amd/sched: move adding finish callback to amd_sched_job_begin"

v2: set an error on the finished fence

Signed-off-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Andres Rodriguez <andresx7@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/sched: NULL out the s_fence field after run_job

amd_sched_process_job drops the fence reference, so NULL out the s_fence
field before adding it as a callback to guard against accidentally using
s_fence after it may have be freed.

v2: add a clarifying comment

Signed-off-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Andres Rodriguez <andresx7@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/sched: move adding finish callback to amd_sched_job_begin

The finish callback is responsible for removing the job from the ring
mirror list, among other things. It makes sense to add it as callback
in the place where the job is added to the ring mirror list.

Signed-off-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Andres Rodriguez <andresx7@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/sched: fix an outdated comment

Signed-off-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Andres Rodriguez <andresx7@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/sched: rename amd_sched_entity_pop_job

The function does not actually remove the job from the FIFO, so "peek"
describes it better.

Signed-off-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Andres Rodriguez <andresx7@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: minor coding style fix

Fix two minor 80 char issues.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: add transparent huge page support for DMA allocations v2

Try to allocate huge pages when it makes sense.

v2: fix comment and use ifdef

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: add support for different pool sizes

Correctly handle different page sizes in the memory accounting.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: remove unsued options from ttm_mem_global_alloc_page

Nobody is actually using that, remove it.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add uvd enc irq

Add UVD encode IRQ handle and enable the UVD encode trap

Signed-off-by: James Zhu <James.Zhu@amd.com>
Reviewed-and-Tested-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add uvd enc ib test

Generate create/destroy messages to test UVD encode indirect buffer function.
And enable UVD encode IB test during device initialization.

Signed-off-by: James Zhu <James.Zhu@amd.com>
Reviewed-and-Tested-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add uvd enc ring test

Add UVD encode ring test functions. And enable UVD encode ring test
during UVD encode hardware initialization.

Signed-off-by: James Zhu <James.Zhu@amd.com>
Reviewed-and-Tested-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add uvd enc vm functions (v2)

Add UVD encode ring vm functions to handle frame ecoding.

v2: squash in warning fix (James)

Signed-off-by: James Zhu <James.Zhu@amd.com>
Reviewed-and-Tested-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add uvd enc into run queue

Signed-off-by: James Zhu <James.Zhu@amd.com>
Reviewed-and-Tested-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add uvd enc rings

UVD 6.3 has two UVD encode rings.  Add the ring structures and initialize the hw ring buffers.
Currently only ASIC Polaris10/11/12 uses UVD6.3 encode engine on HEVC encoding.

Signed-off-by: James Zhu <James.Zhu@amd.com>
Reviewed-and-Tested-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add new uvd enc ring methods

Add new UVD encode ring methods get/set/emit/flush/sync to support uvd6.3 HEVC encoding

Signed-off-by: James Zhu <James.Zhu@amd.com>
Reviewed-and-Tested-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add uvd enc command in header

Add UVD encode command interface definition for uvd6.3 HEVC encoding

Signed-off-by: James Zhu <James.Zhu@amd.com>
Reviewed-and-Tested-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add uvd enc registers in header

Add UVD encode write/read/size/base registers definition for uvd6.3 HEVC ecoding

Signed-off-by: James Zhu <James.Zhu@amd.com>
Reviewed-and-Tested-by: Leo Liu <leo.liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: fix mclk can't switch on Tonga

regression issue caused by
commit 47047263c52779f1f3393c32e3e53661b53a372e
("drm/amd/powerplay: delete eventmgr related files.")

Reviewed-by: Evan Quan <evan.quan@amd.com>
Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay:  Partially revert changes and fix smu7_notify_smc_display()

This partially reverts 0b6b4cbf77c995a34a4ec3d705a636434dadc51a and fixes
the noise issues on Tonga.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: fix typo on avfs disable

Signed-off-by: Evan Quan <evan.quan@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: get raven sclk and mclk levels (v2)

v2: squash in rebase fix (Tom)

Signed-off-by: Evan Quan <evan.quan@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: get raven current sclk and mclk (v2)

v2: squash in rebase fix (Tom)

Signed-off-by: Evan Quan <evan.quan@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: get raven max/min gfx clocks (v2)

v2: squash in rebase fix (Tom)

Signed-off-by: Evan Quan <evan.quan@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: added new raven ppsmc messages

Signed-off-by: Evan Quan <evan.quan@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>