drm/vmwgfx: Explicityly track screen target width and height

We can no longer make the assumption that vmw_stdu_update_st() will
be called when there's a valid display surface attached.  So
instead of using display_srf for width and height, make a record of
these paremeters when the screen target is first defined.

Signed-off-by: Sinclair Yeh <syeh@vmware.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>

drm/vmwgfx: Turn on DRIVER_ATOMIC flag

Now that the legacy path has been tested, turn on the
DRIVER_ATOMIC flag so user mode driver can start going through
the Atomic path.

Signed-off-by: Sinclair Yeh <syeh@vmware.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Acked-by: Daniel Vetter <daniel@ffwll.ch>

drm/vmwgfx: Switch over to internal atomic API for SOU and LDU

Switch over to internal atomic API.  This completes the atomic
internal atomic switch for all the Display Units.

Signed-off-by: Sinclair Yeh <syeh@vmware.com>
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>

drm/vmwgfx: Switch over to internal atomic API for STDU

Switch over to using internal atomic API for mode set.

This removes the legacy set_config API, replacing it with
drm_atomic_helper_set_config().  The DRM helper will use various
vmwgfx-specific atomic functions to set a mode.

DRIVER_ATOMIC capability flag is not yet set, so the user mode
will still use the legacy mode set IOCTL.

v2:
* Avoid a clash between page-flip pinning and setcrtc pinning, modify
the page-flip code to use the page-flip helper and the atomic callbacks.
To enable this, we will need to add a wrapper around atomic_commit.

* Add vmw_kms_set_config() to work around vmwgfx xorg driver bug

Signed-off-by: Sinclair Yeh <syeh@vmware.com>
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>

drm/vmwgfx: Fixes to vmwgfx_fb

1.  When unsetting a mode, num_connector should be set to zero
2.  The pixel_format field needs to be initialized as newer DRM internal
    functions checks this field
3.  Take the drm_modeset_lock_all() because vmw_fb_kms_detach() can
    change current mode

Signed-off-by: Sinclair Yeh <syeh@vmware.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>

drm/vmwgfx: Add and connect atomic state object check/commit

This connects the main state object check and commit function.

v2
* Use drm_atomic_helper_commit() rather than a vmwgfx-specific one

Signed-off-by: Sinclair Yeh <syeh@vmware.com>
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Acked-by: Daniel Vetter <daniel@ffwll.ch>

drm/vmwgfx: Add and connect connector helper function

These helpers won't be called until we flip on the atomic support
flag or set drm_crtc_funcs->set_config to using the atomic
helper.

v2
Use drm_atomic_helper_best_encoder() rather than a vmwgfx-specific one

Signed-off-by: Sinclair Yeh <syeh@vmware.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Acked-by: Daniel Vetter <daniel@ffwll.ch>

drm/vmwgfx: Add and connect plane helper functions

Refactor previous FB and cursor plane update code into their
atomic counterparts: check, update, prepare, cleanup, and disable.

These helpers won't be called until we flip on the atomic support
flag or set drm_crtc_funcs->set_config to using the atomic
helper.

v2:
* Removed unnecessary pinning of cursor surface
* Added a few function headers

v3:
* Set clip region equal to the destination region
* Fixed surface pinning policy
* Enable SVGA mode in vmw_sou_primary_plane_prepare_fb

Signed-off-by: Sinclair Yeh <syeh@vmware.com>
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>

drm/vmwgfx: Add and connect CRTC helper functions

Atomic mode set requires us to refactor existing vmw_stdu_crtc_set_config
code into sections that check the validity of the new mode, and sections
that actually program the hardware state.

vmw_du_crtc_atomic_check() takes CRTC-related checking code.  In a later
patch, vmw_du_primary_plane_atomic_check() will take framebuffer-related
checking code.

These helpers won't be called until we flip on the atomic support
flag or set drm_crtc_funcs->set_config to using the atomic
helper.

v2:
* The state->num_connector is actually the total number of potential
  connectors, not just the one associated with the display unit.
  The proper one to check is ->connector_mask.

* Add the check to only allow plane state to be the same as crtc state
  (Thanks to mlankhorst)

* Make sure to turn on SVGA mode before using VRAM.  SVGA mode is
  disabled in master_drop if dbdev is not running.

v3:
* Moved dot clock override to crtc_atomic_check

Signed-off-by: Sinclair Yeh <syeh@vmware.com>
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Acked-by: Daniel Vetter <daniel@ffwll.ch>

drm/vmwgfx: Connector atomic state

Add connector handling functions. Start tracking is_implicity in
the connector state.  Eventually, this field should be tracked
exclusively in a connector state.

Now that plane and connector states have been created, we can also
activate the code that use CRTC state.

Signed-off-by: Sinclair Yeh <syeh@vmware.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>

drm/vmwgfx: Plane atomic state

Add plane state handling functions.

We have to keep track of a few plane states so we cannot use the
DRM helper for this.

Created vmw_plane_state along with functions to reset, duplicate,
and destroty it.

v2
* Removed cursor clean up special case

Signed-off-by: Sinclair Yeh <syeh@vmware.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>
Acked-by: Daniel Vetter <daniel@ffwll.ch>

drm/vmwgfx: CRTC atomic state

Create and Add CRTC state.  We currently do not track any properties
or custom states so we can technically use the DRM helpers.  Creating
this code just to make potential future additions easier.

Most of the new code will be compiled but not enabled until
plane/connector state handling code is also in place.

This is the first of a series to enable atomic mode set for vmwgfx.

The atomic enabling effort was done in collaboration with Thomas
Hellstrom and the VMware Graphics Team.

Signed-off-by: Sinclair Yeh <syeh@vmware.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>

drm/vmwgfx: Add universal plane support

Universal support is prerequisite for atomic mode set.

Explicitly create planes for the cursor and the primary FB.  With
a functional cursor plane, the DRM will no longer use the legacy
cursor_set2 and cursor_move entry points.

Signed-off-by: Sinclair Yeh <syeh@vmware.com>
Signed-off-by: Thomas Hellstrom <thellstrom@vmware.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>

drm/vmwgfx: Removed unused snooper.crtc field

This field is not being used anymore

Signed-off-by: Sinclair Yeh <syeh@vmware.com>
Reviewed-by: Thomas Hellstrom <thellstrom@vmware.com>

Merge branch 'drm-next-4.12' of git://people.freedesktop.org/~agd5f/linux into drm-next

New stuff for 4.12:
- Preliminary vega10 support
- Support for multi-level page tables
- GPU sensor stuff for mesa
- job tracing improvements
- PRT support for sparse buffers
- Additional SR-IOV improvements
- ttm improvements
- misc bug fixes and code cleanups

* 'drm-next-4.12' of git://people.freedesktop.org/~agd5f/linux: (315 commits)
  drm/amdgpu: Fix 32bit x86 compilation warning
  drm/amdgpu: just disallow reading untouched registers
  drm/amdgpu: remove duplicate allowed reg CP_CPF_BUSY_STAT
  drm/amdgpu/soc15: enable psp block for SRIOV
  drm/amdgpu/soc15: bypass pp block for vf
  drm/amdgpu/psp: add check sOS sign
  drm/amd/amdgpu: Correct ring wptr address in debugfs (v2)
  drm/amdgpu: Fix multi-level page table bugs for large BOs v3
  drm/amdgpu: Fix Vega10 VM initialization
  drm/amdgpu: Make max_pfn 64-bit
  drm/amdgpu: drop GB_GPU_ID from the golden settings
  drm/amdgpu: fix vm pte pde flags to 64-bit for sdma (v3)
  drm/amd/amdgpu: fix Tonga S3 resume hang on rhel6.8
  drm/ttm: decrease ttm bo priority number
  drm/amd/amdgpu: fix performance drop when VRAM pressure
  drm/amdgpu: Couple small warning fixes
  drm/amdgpu: Clean up GFX 9 VM fault messages
  drm/amdgpu: Register UTCL2 as a source of VM faults
  drm/amdgpu/soc15: drop support for reading some registers
  drm/amdgpu/soc15: return cached values for some registers (v2)
  ...

drm/amdgpu: Fix 32bit x86 compilation warning

drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c:187:2: warning: right shift count >= width of type [enabled by default]
drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c:173:2: warning: right shift count >= width of type [enabled by default]
drivers/gpu/drm/amd/amdgpu/vega10_ih.c:106:3: warning: right shift count >= width of type [enabled by default]

v2: Add a space between "&" and "0xff"

Reported by: kbuild-all@01.org

Signed-off-by: Alex Xie <AlexBin.Xie@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: just disallow reading untouched registers

Not sure what the original intention was here, but returning a random piece of
kernel memory to userspace because we didn't set the value at all is clearly
not a good idea.

This patch disallows reading the register and returns
a proper error code instead.

Signed-off-by: Christian König <christian.koenig@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Tom St Denis <tom.stdenis@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: remove duplicate allowed reg CP_CPF_BUSY_STAT

Remove duplicate mmCP_CPF_BUSY_STAT from the allowed registers.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/soc15: enable psp block for SRIOV

SRIOV can support for loading ucode with PSP block, enable it.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Acked-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/soc15: bypass pp block for vf

Disable pp block if device is  vf.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <Monk.Liu@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/psp: add check sOS sign

Confirm if sys driver and sOS are already been loaded through sOS
sign register, skip loading sys driver and sOS if finding the sign.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Acked-by: Huang Rui <ray.huang@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu: Correct ring wptr address in debugfs (v2)

On gfx9 hardware the value is not wrapped and is a 64-bit value.  So
we reduce it modulo the ring size.

Signed-off-by: Tom St Denis <tom.stdenis@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
(v2) use buf_mask instead of computing on the fly

Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Fix multi-level page table bugs for large BOs v3

Fix the start/end address calculation for address ranges that span
multiple page directories in amdgpu_vm_alloc_levels.

Add error messages if page tables aren't found. Otherwise the page
table update would just fail silently.

v2:
 * Change WARN_ON to WARN_ON_ONCE
 * Move masking of high address bits to caller
 * Add range-check for "from" and "to"
v3:
 * Replace WARN_ON_ONCE in get_pt with pr_err in caller

Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Fix Vega10 VM initialization

adev->family is not initialized yet when amdgpu_get_block_size is
called. Use adev->asic_type instead.

Minimum VM size is 512GB, not 256GB, for a single page table entry
in the root page table.

gmc_v9_0_vm_init is called after adev->vm_manager.max_pfn is
initialized. Move the minimum VM-size enforcement ahead of max_pfn
initializtion. Cast to 64-bit before the left-shift.

Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Make max_pfn 64-bit

With 4-level page tables the maximum VM size is 256TB. That's 64G
pages, which can't be represented in 32-bit.

Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: drop GB_GPU_ID from the golden settings

That register is marked deprecated, reading it results in a bus error.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: fix vm pte pde flags to 64-bit for sdma (v3)

v2: fix for all sdma engines
v3: squash in fix for SI/CI

Signed-off-by: Junwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu: fix Tonga S3 resume hang on rhel6.8

1. security firmware loading has moved to sw init, so this code
   is useless.
2. it seems that driver could not call request_firmware on
   kernel 2.6, when S3 resume. for request firmware depends on
   userspace, at this time, userspace is freeze.

Signed-off-by: Jim Qu <Jim.Qu@amd.com>
Acked-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/ttm: decrease ttm bo priority number

decrease and also reserve priority number for KFD using

Signed-off-by: Roger.He <Hongbo.He@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu: fix performance drop when VRAM pressure

When VRAM pressue and trigger huge evictions there is performance drop,
this patch fix it.

Signed-off-by: Roger.He <Hongbo.He@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Couple small warning fixes

Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Clean up GFX 9 VM fault messages

Clean up the VM fault message format and use rate-limiting similar
to other ASICs.

Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Register UTCL2 as a source of VM faults

Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/soc15: drop support for reading some registers

The RB harvest registers are not necessary, the driver already
exposes this info via the info ioctl.  GB_BACKEND_MAP has
been deprecated since SI and is not relevant to the RB mapping.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/soc15: return cached values for some registers (v2)

Required for SR-IOV and saves MMIO transactions.

v2: drop cached RB harvest registers

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gfx9: use hweight for calculating num_rbs

Match what we do for other asics.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: refine the logic in amdgpu_need_post()

We check the mem config register to make sure it's been
programmed by the vbios to determine if we need to post
so we check for a non-0 value.  However, when the asic
comes out of reset, we may see all ones here, so check
for that too.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Add interrupt entries for CRTC_VERTICAL_INTERRUPT0.

This used by DAL ISR logic for VBLANK handling.

Signed-off-by: Andrey Grodzovsky <Andrey.Grodzovsky@amd.com>
Reviewed-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:changes in gfx DMAframe scheme (v2)

1) Adapt to vulkan:
Now use double SWITCH BUFFER to replace the 128 nops w/a,
because when vulkan introduced, umd can insert 7 ~ 16 IBs
per submit which makes 256 DW size cannot hold the whole
DMAframe (if we still insert those 128 nops), CP team suggests
use double SWITCH_BUFFERs, instead of tricky 128 NOPs w/a.

2) To fix the CE VM fault issue when MCBP introduced:
Need one more COND_EXEC wrapping IB part (original one us
for VM switch part).

this change can fix vm fault issue caused by below scenario
without this change:

>CE passed original COND_EXEC (no MCBP issued this moment),
 proceed as normal.

>DE catch up to this COND_EXEC, but this time MCBP issued,
 thus DE treats all following packages as NOP. The following
 VM switch packages now looks just as NOP to DE, so DE
 dosen't do VM flush at all.

>Now CE proceeds to the first IBc, and triggers VM fault,
 because DE didn't do VM flush for this DMAframe.

3) change estimated alloc size for gfx9.
with new DMAframe scheme, we need modify emit_frame_size
for gfx9

4) No need to insert 128 nops after gfx8 vm flush anymore
because there was double SWITCH_BUFFER append to vm flush,
and for gfx7 we already use double SWITCH_BUFFER following
after vm_flush so no change needed for it.

5) Change emit_frame_size for gfx8

v2: squash in BUG removal from Monk

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:fix missing programing critical registers

those MC_VM registers won't be programed by VBIOS in VF
so driver is responsible to programe them.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:fix ring_write_multiple

ring_write_multiple should use buf_mask instead of ptr_mask

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:fix gmc_v9 vm fault process for SRIOV

for SRIOV we cannot use access register when in IRQ routine
with regular KIQ method

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:no cg for soc15 of SRIOV

no CG for SRIOV on SOC15

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:two fixings for sdma v4 for SRIOV

no hw_fini for SRIOV, otherwise other VF will be affected
no CG for SRIOV

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:change sequence of SDMA v4 init

must set minor_update.enable before write smaller value
to wptr/doorbell, so for sriov we need set that register
bit in hw_init period.

this could fix the SDMA ring test fail after guest reboot

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:fix ring init sequence

ring->buf_mask need be set prior to ring_clear_ring invoke
and fix ring_clear_ring as well which should use buf_mask
instead of ptr_mask

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:virt_init_setting invoke is missed!

this must be invoked during early init

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

uapi/drm:add new flag for Preemption

when MCBP supported, we will set pre_enb bit for those
IBs with PREEMPT flag tagged

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

uapi/drm:change Preamble Description

Preamble in linux doesn't mean it is CE PREAMBLE IB,
instead it means this IB could be dropped if no
ctx switch happens.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:enable MCBP for SR-IOV (v2)

Apply the new IB during IB emit for SRIOV with MCBP

v2: agd: use define instead of magic number

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:enable mcbp for gfx9(v2)

set bit 21 of IB.control filed to actually enable
MCBP for SRIOV
v2:
add flag for preemption enable bit for soc15 and use
this flag instead of hardcode.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:implement cond_exec for gfx8

when MCBP enabled for gfx8, the cond_exec must also
be implemented, otherwise there will be odds to meet
cross engine (ce and me) deadlock when world switch
happens.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:fix the check in cs_ib_fill for SRIOV

1,the check is only appliable for SRIOV GFX engine.
2,use chunk_ib instead of ib.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Ken Wang <Qingqing.wang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:protect cs submit

to prevent submit two or more IBs with PREEMPT flags.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:fix cs_ib_fill

should use chunk_ib instead of ib, otherwise the logic
is incorrect.

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Ken Wang <Qingqing.wang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: clear freed mappings immediately when BO may be freed

Also, add the fence of the clear operations to the BO to ensure that
the underlying memory can only be re-used after all PTEs pointing to
it have been cleared.

This avoids the following sequence of events that could be triggered
by user space:

1. Submit a CS that accesses some BO _without_ adding that BO to the
   buffer list.
2. Free that BO.
3. Some other task re-uses the memory underlying the BO.
4. The CS is submitted to the hardware and accesses memory that is
   now already in use by somebody else.

By clearing the page tables immediately in step 2, a GPU VM fault will
be triggered in step 4 instead of wild memory accesses.

v2: use amdgpu_bo_fence directly

Signed-off-by: Nicolai Hähnle <nicolai.haehnle@amd.com>
Reviewed-by: Chunming Zhou <david1.zhou@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: enable four level VMPT for gmc9

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: set page table depth by num_level

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: adapt vm size for multi vmpt

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: limit block size to one page

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: abstract block size to one function

Signed-off-by: Chunming Zhou <David1.Zhou@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add alloc/free for multi level PDs V2

Allocate and free page directories on demand.

V2:
a. clear entries allocation
b. fix entries index calculation
c. need alloc sub level even parent bo was allocated

Signed-off-by: Christian König <christian.koenig@amd.com> (v1)
Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (v1)
Signed-off-by: Chunming Zhou <David1.Zhou@amd.com> (v2)
Acked-by: Alex Deucher <alexander.deucher@amd.com> (v2)
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: handle multi level PD during PT updates

Not the best solution, but good enough for now.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: handle multi level PD updates V2

Update all levels of the page directory.

V2:
a. sub level pdes always are written to incorrect place.
b. sub levels need to update regardless of parent updates.

Signed-off-by: Christian König <christian.koenig@amd.com> (V1)
Reviewed-by: Alex Deucher <alexander.deucher@amd.com> (V1)
Signed-off-by: Chunming Zhou <David1.Zhou@amd.com> (V2)
Acked-by: Alex Deucher <alexander.deucher@amd.com> (V2)
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: handle multi level PD in the LRU

Move all levels to the end after command submission.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: handle multi level PD during validation

All page directory levels should be in place after this.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: handle multi level PD size calculation (v2)

Allows us to get the size for all levels as well.

v2: agd: fix warning

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: generalize page table level

No functional change, but the base for multi level page tables.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add num_level to the VM manager

Needs to be filled with handling.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add the VM pointer to the amdgpu_pte_update_params as well

This way we save passing it through the different functions.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: rename page_directory_fence to last_dir_update

Decribes better what this is used for.

Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/powerplay: reduce sample period time

for power readings.

Signed-off-by: Rex Zhu <Rex.Zhu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add get_clockgating callback for mmhub v1

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add get_clockgating for sdma v4

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add get_clockgating callback for soc15 (v3)

v2: squash register typo fix from Ray
v3: fix spelling

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add get_clockgating callback for nbio v6.1

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add get_clockgating callback for gfx v9

Signed-off-by: Huang Rui <ray.huang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Edward O'Callaghan <funfunctor@folklore1984.net>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gfx9: further KIQ parameter cleanup

The ring structure already has what we need.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gfx9: store the eop gpu addr in the ring structure

Avoids passing around additional parameters during setup.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gfx9: reduce the functon params for mpq setup

Everything we need is in the ring structure.  No need to
pass all the bits explicitly.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gfx9: reserve kiq eop object before unmapping it

It's required.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gfx9: reserve mqd objects before mapping them

It's required.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gfx9: rename some functions

To better match where they are used.  Called from sw_init
and sw_fini.

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gfx9: whitespace cleanup

Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amd/amdgpu: Fix some warnings in vce4

Signed-off-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/vce4: impl vce & mmsch sriov start

For MM sriov, need use MMSCH to init engine and the init procedures
are all saved in mm table.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: add mmsch structures

For MM SRIOV, need to prepare MM table send send it to MMSCH to
initial UVD & VCE engine. Create new header file for the structures.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/vce4: Ignore vce ring/ib test temporarily

In order to not break SRIOV gfx development, will revert
this patch after vce proved working.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/vce4: alloc mm table for MM sriov

Allocate MM table for sriov device.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/virt: add structure for MM table

Add new structure for MM table for multi media scheduler of sriov.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <Monk.Liu@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: disable uvd for sriov

disable uvd for sriov temporarily.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/vce4: enable doorbell for SRIOV

VCE SRIOV need use doorbell and only works on VCN0 ring now

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu: Don't touch PG&CG for SRIOV MM

For SRIOV, MM don't need to care about PG & CG, skip it.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/vega10:fix DOORBELL64 scheme

Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu:vega10: enable virtual display if set via module option

Enable virtual displays if the user has enabled them via the
kernel command line.  Useful in virtual or headless environments.

Reviewed-by: Harry Wentland <harry.wentland@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/soc15: enable virtual dce for vf

VF need virtual dce, enable it if device is vf.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <Monk.Liu@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/soc15: init virt ops for vf

If gpu device is vf, set virt ops so that guest can talk with GPU
hypervisor.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <Monk.Liu@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/virt: impl mailbox for ai

Implement mailbox protocol for AI so that guest vf can communicate
with GPU hypervisor.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Monk Liu <Monk.Liu@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/dce_virtual: bypass DPM for vf

If enable DPM for VF, always get lot of warn_slow_patch_null in
dmesg and vf doesn't support DPM.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>

drm/amdgpu/gmc9: no need use kiq in vega10 tlb flush

two reasons:
1. there is a spinlock around;
2. vm register is pf/vf copy, vf can access via mmio safely.

Signed-off-by: Xiangliang Yu <Xiangliang.Yu@amd.com>
Signed-off-by: Monk Liu <Monk.Liu@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>